Key Concepts
Share
A Manila share is the fundamental resource allocated by the Shared File System
service. It represents a persistent, readable, and writable file system that
can be accessed by OpenStack compute instances or by clients outside of
OpenStack, depending on the deployment configuration. The connection between
the share consumer and the Manila service can use various protocols, including
NFS and CIFS. Protocol support depends on the deployed Manila driver and the
end user’s selection.
Warning
A Manila share is an abstract storage object that may or may not directly
map to a “share” concept from the underlying storage backend.
Manila shares are uniquely identified by a UUID assigned at creation. A share
may also have a human-readable name, but this name is not guaranteed to be
unique within a tenant or Manila deployment.
The actual capacity provisioned for a Manila share resides on a single Manila
backend within a single resource pool.
Backend
A Manila backend is a configuration object that represents a single provider of
resource pools used to fulfill provisioning requests for shared file systems. A
Manila backend communicates with the storage system through a Manila driver.
Multiple backends can be configured and managed simultaneously, even when using
the same Manila driver.
Note
A single Manila backend may be defined in the [DEFAULT] stanza of
manila.conf. However, NetApp recommends setting the
enabled_share_backends configuration option to a comma-separated list of
backend names, with each backend having its own configuration stanza named as
listed in the enabled_share_backends option. See the section
“Theory of Operation: Manila.conf” for an example.
Storage Pools
With the Kilo release of OpenStack, Manila introduced the concept of
“storage pools.” The backend storage may present one or more logical storage
resource pools from which Manila selects a storage location when provisioning
shares. In releases prior to Kilo, NetApp’s Manila drivers included logic to
determine which aggregate a share would be placed in. With the introduction of
pools, all scheduling logic is handled entirely by the Manila scheduler.
Important
For NetApp’s Manila drivers, before the Xena release, a Manila storage pool
is an aggregate defined within ONTAP. From the Xena release onward, the
storage pool can be either an aggregate (for FlexVol provisioning) or a set
of aggregates (for FlexGroup provisioning; see Deployment Choice: Utilizing FlexGroup Pool).
Driver
A Manila driver is a specific implementation of a Manila backend that maps the
abstract APIs and primitives of Manila to the appropriate constructs within the
underlying storage solution.
Caution
The term “driver” can be confusing, as it is often associated with device
drivers in operating systems, which provide a data I/O path. In Manila,
driver implementations provide provisioning and management of storage
devices, but do not sit in the data I/O path. For this reason, “driver” is
sometimes used interchangeably with the term “provider.”
A Manila share type is an abstract collection of criteria used to characterize
Manila shares. Share types are commonly used to create a hierarchy of
functional capabilities that represent different tiers of storage service. For
example, a cloud administrator might define a premium share type for higher
performance and a basic share type for best-effort performance.
The criteria are specified as key/value pairs, which the Manila scheduler
inspects to determine which resource pools can fulfill a provisioning request.
Individual drivers and backends may advertise arbitrary key/value pairs
(capabilities) to the scheduler for each pool. These are compared against share
type definitions to determine which pool will fulfill a provisioning request.
Snapshot
A Manila snapshot is a point-in-time, read-only copy of a Manila share.
Snapshots can be created from an existing share that is operational,
regardless of whether a client has mounted the file system. A snapshot can
serve as the content source for a new share when the share is created with the
create from snapshot option.
Important
In the Mitaka and Newton releases of OpenStack, snapshot support is enabled
by default for newly created share types. Beginning with the Ocata release,
the snapshot_support extra spec must be set to True to allow
snapshots for a share type. If the snapshot_support extra spec is
omitted or set to False, users cannot create snapshots on shares of that
type.
Other snapshot-related extra specs in the Ocata release (and later) include:
If an extra spec is left unset, it defaults to False. A newly created
share may or may not end up on a backend with the associated capability.
Set the extra spec explicitly to False if you want shares created only
on backends that do not support the associated capabilities. For a table of
NetApp supported extra specs, refer to Table 6.10,
“NetApp supported Extra Specs for use with Manila Share Types”.
Important
From the Ussuri release, a share can be created from a snapshot and placed
in a pool or backend different from the parent share. To allow shares to be
placed in other pools or backends, set
use_scheduler_creating_share_from_snapshot=True in the Manila
configuration file. To perform this operation across clusters, the
replication feature must be enabled and the backend property
replication_domain must be configured. More details are available
here.
Share Group
A Manila share group is a construct that allows you to group shares together.
Share groups enable actions on multiple shares at once, such as creating
consistent, point-in-time snapshots. Share group snapshots can be created from
an existing share group, and all shares in a share group snapshot can be
restored by creating a share group from that snapshot.
Note
When using NetApp’s Manila drivers before the Xena release, Share Groups are
synonymous with the conventional Consistency Group construct. From Xena
onward, if a FlexGroup pool is configured, consistency group features are
not supported, and Share Groups will not have the features of a
Consistency Group. Beginning with the Rocky release, OpenStack
recommends using Share Groups to create a grouping construct for operating
on groups of shares.
Note
All shares in a share group must be on the same share network and share
server.
Warning
From the Xena release, configuring a FlexGroup pool removes consistency
group support for the entire backend. Even FlexVol pools lose this support.
Share Access Rules
Share access rules define which clients can access a particular Manila share.
For NFS shares, access rules are declared by listing valid IP networks (using
CIDR notation) that should have access to the share. For CIFS shares, the
Windows security identifier (SID) can be specified.
Important
For NetApp’s Manila drivers, share access is enforced through export
policies configured on the NetApp storage controller.
Warning
There is a known issue when adding several access rules in quick succession.
The share instance access-rules-status may change to “updating multiple” on
the API after the manager has already checked for this status. This can
cause the allow/deny APIs to become stuck for that share instance. If this
occurs, there are two workarounds:
Security Services
Security services in Manila allow finer-grained client access rules for
authentication and authorization to share content. External services,
including LDAP, Active Directory, and Kerberos, can be declared as
resources that are consulted when access decisions are made for a
particular share. Shares can be associated with multiple security
services.
Important
When creating a CIFS share, users must create a security service with one
of three options (LDAP, Active Directory, or Kerberos) and then add that
security service to an existing share network.
Important
Beginning with the Wallaby release, Manila can add or update security
services that are currently in use. Updating a security service requires
both the old and new security services to have the same type. When adding
a security service to an in-use share network, make sure the share network
does not already contain a security service of the same type; otherwise,
the operation is denied. For more details, refer to the
security service modification session
Share Networks
A share network defines the relationship between a tenant’s networks or subnets
(as defined in an OpenStack network service such as Neutron or Nova-network)
and the Manila shares created by that tenant. This allows a tenant to provision
shares so that only instances connected to a specific OpenStack-defined network
can access the share.
Note
As of the Kilo release, share networks are no longer required arguments when
creating shares.
Share Network Subnets
A share network subnet defines the relationship between a single tenant’s
network or subnet (as defined in an OpenStack network service such as Neutron)
and the Manila share instance created by that tenant. Since the Train release,
information that previously belonged to the share network entity is now part of
the share network subnet object. As a result, the share server object is now
associated with a share network subnet instead of a share network.
The share network subnet is defined for each Availability Zone (AZ) during
share network creation. From the Yoga release, a single AZ of the share network
can be configured with multiple share network subnets, provided they are on the
same VLAN segment.
Share Servers
A share server is a logical entity that manages shares created on a specific
share network. Depending on the Manila driver implementation, a share server
may be a configuration object within the storage controller or represent
logical resources provisioned within an OpenStack deployment to support the
data path for accessing Manila shares.
Share servers interact with network services to determine the appropriate IP
addresses for exporting shares according to the related share network. Manila
uses a pluggable network model, allowing share servers to work with OpenStack
environments that use either Nova-Network or Neutron. Manila also includes a
standalone network plugin that manages a pool of IP addresses for shares,
defined in the manila.conf file. For more details on how share servers
interact with network services, see Figure 6.2. Manila Workflow - Share Creation with Share Servers and Figure 6.3. Manila Workflow - Share Creation without Share Servers.
Important
In the NetApp Manila driver, a share server is defined as a storage virtual
machine (SVM, also known as a Vserver) within the ONTAP system, associated
with a particular backend. The NetApp Manila driver operates in two modes:
Dynamic creation of share servers (SVMs) for each share network—this is
called the
NetApp Manila driver with share server management.
Reuse of a single share server (SVM) for all shares hosted from a backend—
this is called the NetApp Manila driver without share server
management.
Important
Beginning with the Stein release, Manila supports managing and unmanaging
share servers. This allows importing share servers and managing them with
Manila. For a detailed example, see
Importing and exporting Manila Share servers.
Note
From the Train release, the share server is associated with a share network
subnet instead of a share network.
Share Replicas
Share replicas are a way to mirror share data to another storage pool so that
the data is stored in multiple locations to allow failover in a disaster
situation. Manila currently allows three types of replication: writable,
readable, and DR.
Writable - Synchronously replicated shares where all replicas are writable.
Promotion is not supported and not needed.
Readable - Mirror-style replication with a primary (writable) copy and one
or more secondary (read-only) copies which can become writable after a
promotion of the secondary.
DR (for Disaster Recovery) - Generalized replication with secondary copies
that are inaccessible. A secondary replica will become the primary replica,
and accessible, after a promotion.
Important
The NetApp Unified Driver for ONTAP provides DR replication only if you are
operating without Share Server management until Stein Release. From Train
release, the NetApp Unified Driver for ONTAP with Share Server management
does support DR style replication. From Xena release, the NetApp Unified
Driver also supports the Readable replication type. In older releases, only
the DR type was supported.
Share Migration
Beginning with the Ocata release, NetApp’s Manila driver supports
non-disruptive migration of Manila shares - along with the filesystem
metadata and snapshots, if desired. This can be useful in a variety of
use cases, such as during maintenance or evacuation.
Share migration is a two-step process which includes starting the
migration process using the manila migration-start command, and then
completing the process using the manila migration-complete command.
For the list of migration commands, refer to
“Table 6.9, “Manila API Overview - Share Migration”.
Caution
FlexGroup shares cannot be migrated.
Share Management
Managing and unmanaging of shares is an admin-only operation that makes it
possible to control the visibility of shared filesystem storage with respect
to Manila. Managing a share refers to registering a non-Manila share with its
size, shared filesystem protocol, share server, and export path into Manila
management. Unmanaging a share refers to unregistering a Manila share and
removing it from Manila’s database. The unmanage option can be reverted,
thus making it possible to import the share to Manila control if desired.
Important
When managing a share, Manila assigns a new UUID to the share and renames
the share on the storage backend to the following format:
share_<share_instance_id>. This ensures that the same share cannot be
managed twice.
Share Server Management
Since share servers are logical containers that associate shares with share
networks, there is also a provision to manage and unmanage them. Admin users
can manage share servers by specifying the host, share network, and the
driver-specific identifier that are used to uniquely identify the share
server. This functionality helps in importing share servers and their
associated shares and snapshots that were created in another system or
deployment. Similarly, share servers can also be unmanaged and removed from
Manila’s database.
Important
When managing a share server, Manila assigns a new UUID to the share server
and renames the share server on the storage backend to the following
format: os_<id>, where <id> is the newly assigned UUID. This ensures
that the same share server cannot be managed twice.
Share Server Migration
Beginning with the Victoria release, NetApp’s Manila driver supports migration
of Manila share servers, including their shares, snapshots, and access rules.
This is useful for backend evacuation or optimization.
As with share migration, a two-phase approach is available. Administrators can
control when to complete the operation, which ends with client disruption. To
migrate a share server, start the migration, then complete it after the first
phase is done and at a time when client disruption is acceptable. For a list
of all share server migration commands, see
“Table 6.10, “Manila API Overview - Share Server Migration”.
Important
To enable share server migration across ONTAP clusters, the clusters must be
peered in advance and have a valid SnapMirror license installed.
Share Server Migration With SVM DR
NetApp’s Manila driver supports share server migration with SVM DR when all of
the following requirements are met:
The source and destination back ends are in different ONTAP clusters.
Both clusters are running the same ONTAP version (9.4 or later).
- The source and destination share networks have identical security service
configurations.
- The source and destination back ends have compatible capabilities (such as
FlexVol Encryption, SnapRestore license, etc.).
- The destination back end has sufficient free space for the source share
server, including all shares and snapshots.
The migration may be disruptive to client access.
The share server does not contain any FlexGroup shares.
Share Server Migration With SVM Migrate (Available From Xena Release)
SVM Migrate is a NetApp-specific mechanism that allows administrators to move
entire share servers from one cluster to another efficiently. This mechanism
supports non-disruptive migrations, as long as no network changes are
perceived when specifying a new share network during migration start.
NetApp’s Manila driver supports share server migration using SVM Migrate
(from Xena release) if all the following requirements are met:
Source and destination back ends belong to different clusters.
- Source and destination clusters have the same ONTAP system version (9.10 or
higher).
- Source and destination share networks do not have different security service
configurations.
- Source and destination back ends have compatible capabilities (FlexVol
Encryption, SnapRestore license, etc.).
- Destination back end has enough free space for the source share server and
all its shares and snapshots.
The share server does not have NFS version 4.0 enabled.
Both source and destination clusters are AFF.
- Both source and destination clusters contain a pre-created policy of type
‘migrate’.
Share Backup (Available from Caracal Release)
The Bobcat release introduced a new NFS-based generic backup driver for Manila.
This driver is configured on the Manila data service node and provides the
following core capabilities:
In the Caracal release, NetApp extended this solution for NetApp-specific
shares. The NetApp unified driver now supports creating, editing, deleting,
and restoring backups using SnapVault relationships. This approach leverages
ONTAP features such as snapshots, SVM peering within and across ONTAP
clusters, and the ONTAP replication engine.
Share Server and Share Encryption with Barbican (Available from Flamingo Release)
NetApp’s Manila driver supports share encryption using the Barbican keystore.
This feature provides the following basic workflows:
Note
Encryption key management with NetApp is only supported when DHSS=True.
See this for configuration
details in manila.conf.
Important
Follow these steps to create an
encrypted share. Deleting an encrypted share follows the standard share
deletion process.
Important
NetApp manages user-defined encryption keys only at the share server (SVM)
level, not at the volume (share) level. Encryption is enabled for shares
and protected by the share server using the user-supplied Barbican
encryption key.
Synchronous SnapMirror-Based Share Replication (Available from Gazpacho Release)
Beginning with the Gazpacho release, NetApp’s Manila driver supports
user-specified SnapMirror policies for share replication. Previously, share
replica creation always used the MirrorAllSnapshots asynchronous policy.
With this enhancement, users can specify one of the following predefined
SnapMirror policies when creating a share replica using the
replication_policy metadata property:
MirrorAllSnapshots – Asynchronous policy. Data is replicated on a
schedule (default: hourly). This is the default if no policy is specified.
Sync – Synchronous policy. Writes are acknowledged only after being
mirrored to the destination. Provides zero RPO.
StrictSync – Synchronous policy with strict consistency. Writes fail if
the destination is unreachable. Provides zero RPO.
Supported workflows for all policies include: Create, Promote, Resync, and
Delete share replicas.
Note
The replication_policy metadata property is optional and only applies
when replication is enabled. If no policy is specified, the default
MirrorAllSnapshots policy is used, maintaining backward compatibility
with the Flamingo release.
Note
Supported configurations include DHSS True and False, NFS and CIFS
protocols, ZAPI and REST interfaces, FlexVol volumes (all policies),
FlexGroup volumes (async policies only), within-cluster and cross-cluster
replication, and readable and dr replication types. ONTAP versions
9.17.1 and 9.18.1 are qualified.
Important
For asynchronous policies (MirrorAllSnapshots), the SnapMirror
schedule is determined using the following priority:
netapp_snapmirror_schedule from the destination backend stanza
in manila.conf if specified.
Default schedule defined in the SnapMirror policy.
If neither is set, defaults to hourly.
For synchronous policies (Sync, StrictSync), schedule is not
applicable.
Important
FlexGroup shares support only asynchronous SnapMirror replication
(MirrorAllSnapshots). Attempts to use Sync or StrictSync
policies with FlexGroup shares will fail during replica creation.
Important
ONTAP does not permit modification of an existing SnapMirror relationship
policy or mode. To change the policy, users must delete the existing share
replica and create a new one.
Important
Only the predefined policies (MirrorAllSnapshots, Sync,
StrictSync) are supported. Custom ONTAP policies are not supported.
The NetApp driver does not act on share replica set/unset operations for
replication_policy.
Important
ONTAP only allows creation of Sync or StrictSync SnapMirror
relationships when the CIFS SMB1 option is disabled. This applies to both
NFS and CIFS volumes.
Important
Snapshots created before replica creation will not be replicated to the
destination for Sync and StrictSync policies (ONTAP limitation).
The NetApp driver will reject replica creation with Sync or
StrictSync policies if pre-existing snapshots are detected on the
share. For shares that already have snapshots, use the
MirrorAllSnapshots (asynchronous) policy, which supports replication of
pre-existing snapshots.
Important
ONTAP does not support SVM migration when synchronous SnapMirror volumes
exist on the SVM. As a result, Manila share server migration is not
supported for share servers containing shares with Sync or
StrictSync replicas. Refer to the ONTAP SVM data mobility
documentation for
details on supported and unsupported features.
Important
ONTAP does not support SnapMirror Synchronous relationships on FlexClone
volumes that have not been split from their parent. In Manila, when a share
is created from a snapshot (create share from snapshot), the resulting
volume is a FlexClone if the destination is on the same storage cluster. If
the parent share already has a Sync or StrictSync replica,
creating a synchronous replica for such a cloned share will fail during
SnapMirror initialization. This limitation does not apply to the
MirrorAllSnapshots (asynchronous) policy.
To work around this, split the FlexClone volume from its parent on
ONTAP before creating a synchronous replica.
Important
Follow these steps for CLI
examples of creating share replicas with SnapMirror policies.
NetApp Aggregate Encryption support (NAE) (Available from Gazpacho Release)
NetApp’s Manila driver supports aggregate-level encryption using NetApp
Aggregate Encryption (NAE) in ONTAP. Unlike NVE (NetApp Volume Encryption),
which encrypts data at the volume level, NAE encrypts data at the aggregate
level, providing a more efficient approach for multiple shares.
A new capability, netapp_aggregate_encryption, is added to Manila pools
to indicate whether the aggregate is encrypted with NAE. When a share type
specifies the extra spec netapp_aggregate_encryption=True, the Manila
scheduler attempts to place shares on pools that have this capability set to
True.
By default, the driver attempts to create NAE shares on NAE-enabled
aggregates, even if the share type does not specify the extra spec
netapp_aggregate_encryption=True. NAE volume creation therefore depends
on the pool where a share is created, and the
netapp_aggregate_encryption share type extra spec ensures the scheduler
places shares on the correct pool.
To create shares with NAE encryption, the share type must specify the extra spec netapp_aggregate_encryption=True.
Supported workflows include:
Creating shares with NAE encryption on NAE-enabled aggregates
Migrating shares between encrypted and non-encrypted pools
Creating share replicas and promoting replicas between encrypted and
non-encrypted pools
Creating snapshots and creating shares from snapshots between encrypted and
non-encrypted pools
Share server migration
FlexGroup support with NAE encryption
NAE on SVM-scoped backends is not supported
NAE support alongside NVE or share encryption with Barbican is not
supported. However, an NVE share can be created on an NAE aggregate if
the share type specifies only netapp_flexvol_encryption=True.
Note
NAE must be enabled on the ONTAP aggregate as a prerequisite. NetApp
recommends
using cluster-scoped credentials in the backend stanza for NAE support.
Note
ONTAP 9.6 or later is required for NAE support.
Important
A share type must specify either netapp_aggregate_encryption=True or
netapp_flexvol_encryption=True, but not both. Attempting to create a share
type with both encryption specifications will cause share creation to fail.
Similarly, the NAE feature cannot be used alongside share encryption with
Barbican. Attempting to create a share type with both NAE and share
encryption specifications
will cause share creation to fail.
Important
All aggregates in a FlexGroup pool must be either encrypted or
non-encrypted. A mix of encrypted and non-encrypted aggregates is not
supported in a FlexGroup pool.
QoS type specs support (Available from Gazpacho Release)
Beginning with the Gazpacho release, the NetApp ONTAP driver supports QoS
types and their associated specs to provision shares with performance
policies. A QoS type is created with a unique name (maximum length of 60
characters) and contains a set of key-value pairs (specs) that define the
QoS policy. These specs are interpreted by the NetApp driver and translated
into ONTAP QoS policy groups during share provisioning.
The spec with key policy_type is mandatory and must have one of the
following values:
Depending on the selected policy_type, the following spec keys are
supported.
For fixed QoS policy:
max_throughput_iops: Maximum IOPS limit.
max_throughput_mbps: Maximum throughput in MB/s.
min_throughput_iops: Minimum IOPS guarantee.
min_throughput_mbps: Minimum throughput guarantee in MB/s.
capacity_shared: Indicates whether the QoS policy can be shared. Possiblevalues are true or false. The default value is false.
For adaptive QoS policy:
expected_iops: Expected IOPS per workload.
peak_iops: Peak IOPS limit.
absolute_min_iops: Absolute minimum IOPS.
block_size: Block size used for QoS calculation.
peak_iops_allocation: Allocation type for peak IOPS.
expected_iops_allocation: Allocation type for expected IOPS.
Note
To apply a QoS policy to shares, the QoS type must be associated with a
share type using the default_qos_type extra spec. When a share is
created using such a share type, a corresponding QoS policy group is
created on the share server (if not already present). The policy is then
applied to the share. A unique QoS policy name is generated per share
server. If default_qos_type is not defined in the share type, the
share is created without any QoS policy.
Note
If QoS type specs are updated by the cloud administrator:
The updated values are validated by the driver during spec consideration.
The corresponding ONTAP QoS policy is updated.
Changes take effect for shares created after the modification.
Note
QoS type must include the mandatory policy_type spec.
Invalid or unsupported spec keys are rejected during validation, and
share creation fails.
Backend pools must support QoS for scheduling to consider them.
The existing method of creating fixed QoS policies using share-type
extra specs remains supported in this release. However, a share type
can use either share-type extra specs or QoS type specs, but not both
simultaneously. Such combinations are invalid, and share creation fails.