Theory of Operation: Driver Filter and Weigher for Scheduler¶

Overview¶

Cinder provides the DriverFilter and GoodnessWeigher to exercise more fine grained control when choosing a volume backend based on backend specific properties. It is possible to customize the backend definition to schedule the creation of Cinder volumes on specific backends, based on the volume properties and the backend specific properties. Refer to the “Configuration” section to take a look at how this would work.

Setup¶

To enable the driver filter, set the scheduler_default_filters option in the [DEFAULT] section of the cinder.conf file to include DriverFilter.

To enable the goodness filter as a weigher, set the scheduler_default_weighers option in the [DEFAULT] section of the cinder.conf file to GoodnessWeigher or add it to the list if other weighers are already present.

Example cinder.conf configuration file:

[DEFAULT]
scheduler_default_filters = DriverFilter,AvailabilityFilter,CapabilityFilter,CapacityFilter
scheduler_default_weighers = GoodnessWeigher

You can choose to use the DriverFilter without the GoodnessWeigher or vice-versa. The filter and weigher working together, however, create the most benefits when helping the scheduler choose an ideal back end.

Important

The GoodnessWeigher can be used along with CapacityWeigher and others, but must be used with caution as it might obfuscate the CapacityWeigher.

Refer to Upstream Driver Filter Weighing which contains a more detailed explanation on how to use the driver filter and goodness weigher to customize the scheduling mechanism.

There exist three property sets that can be referenced in the filter_funtion and goodness_function definitions:

Host stats for a backend: These refer to the parameters that are specific for the host containing a defined backend. This would be analogous to an ONTAP/SolidFire cluster. They can be invoked as stats.<property>. For example: stats.allocated_capacity_gb. “Table 4.12” lists the host stats that can be used for ONTAP and SolidFire systems.

Property	Type	Products Supported	Description
`host`	String	ONTAP,SolidFire	The name of the host.
`volume_backend_name`	String	ONTAP,SolidFire	The volume backend name as defined in `cinder.conf`
`vendor_name`	String	ONTAP,SolidFire	The name of the vendor. For ONTAP systems, this is `NetApp`. For SolidFire systems, this is `SolidFire Inc`
`driver_version`	String	ONTAP,SolidFire	The Cinder driver version.
`storage_protocol`	String	ONTAP,SolidFire	The storage protocol supported. For ONTAP pools, this depends on the protocol enabled for the storage pool (can be iSCSI, NFS or FC). For SolidFire pools, this is `iSCSI`.
`QoS_support`	Boolean	ONTAP,SolidFire	Boolean signifying whether QoS is supported. This is reported as `True` for ONTAP and SolidFire systems.
`total_capacity_gb`	Float	ONTAP,SolidFire	The total capacity of each pool in GB.
`allocated_capacity_gb`	Float	ONTAP,SolidFire	The capacity that has been allocated for Cinder volumes on the cluster. This value is reported in GB.
`free_capacity_gb`	Float	ONTAP,SolidFire	The amount of free capacity of each pool in GB.
`reserved_percentage`	Float	ONTAP,SolidFire	The reserved percentage for each pool. This specifies how much space is to be reduced from total_capacity when performing over subscription calculations.

Table 4.12 Host statistics for ONTAP and SolidFire systems

Requested volume properties: These statistics are used to control scheduling based on the specifications of the volume requested during creation. These parameters can be referenced as volume.<property>. For example, the size of the requested volume is returned by volume.size. “Table 4.13” contains a list of statistics that can be queried from a volume request. These statistics would be supported for all NetApp backends.

Property	Description
`size`	The size of the volume requested.
`volume_type_id`	The UUID of the volume type.
`display_name`	Name of the volume.
`volume_metadata`	Metadata associated with the volume.
`user_id`	User ID of the user that creates the volume.
`display_description`	Display description for the volume.
`source_volid`	ID of the source volume, if the volume is cloned.
`reservations`	Any reservation the volume has.
`attach_status`	The attach status for the volume.
`id`	The volume’s ID.
`replication_status`	The volume’s replication status.
`snapshot_id`	The volume’s snapshot ID.
`encryption_key_id`	The volume’s encryption key ID.
`volume_admin_metadata`	Any admin metadata for this volume.
`source_replicaid`	The source replication ID.
`consistencygroup_id`	The consistency group ID.
`metadata`	General metadata.

Table 4.13 Volume properties available for Filter and Goodness functions

Important

The most commonly used volume.<property> is volume.size. This enables admins to schedule volume placement based on the size of the volume that is requested.

Backend specific capabilities: The following table contains a list of capabilities reported by the ONTAP and SolidFire Cinder drivers.

Property	Type	Products Supported	Description
`netapp_aggregate_used_percent`	String	ONTAP	The percentage of usage for the aggregate.
`utilization`	String	ONTAP	Node utilization percentage per pool (FlexVol).
`netapp_dedupe_used_percent`	String	ONTAP	The percentage of shared block limit that has been consumed by dedupe and cloning operations.
`max_over_subscription_ratio`	String	ONTAP,SolidFire	The over-provisioning factor when thin provisioning is being used in the storage pool.
`thin_provision_percent`	String	SolidFire	The percentage of thin provisioning performed on the SolidFire cluster.
`compression_percent`	String	SolidFire	The percentage of compression performed on the SolidFire cluster.
`deduplicaton_percent`	String	SolidFire	The percentage of deduplication achieved on the SolidFire cluster.
`provisioned_capacity_gb`	String	SolidFire	Amount of capacity that has been provisioned on the entire SolidFire cluster (not limited to storage consumed by Cinder volumes). Reported in GB.
`provisioned_iops`	String	SolidFire	The amount of provisioned IOPS for the entire SolidFire cluster.
`current_iops`	String	SolidFire	Current active IOPS for the entire SolidFire cluster.
`average_iops`	String	SolidFire	Average IOPS returned for the entire SolidFire cluster.
`max_iops`	String	SolidFire	Maximum IOPS value for the SolidFire cluster.
`peak_iops`	String	SolidFire	Peak IOPS value for the SolidFire cluster.

Table 4.14 Backend capabilities reported by ONTAP and SolidFire Cinder drivers

Configuration¶

To utilize the driver filter and goodness weigher, update the scheduler_default_filters and scheduler_default_weighers options in cinder.conf. The required filter_function and goodness_function are defined on a per-backend basis as shown below.

Example1: Using Driver Filter

[default]
.
.
scheduler_default_filters = DriverFilter,AvailabilityFilter,CapabilityFilter,CapacityFilter
scheduler_default_weighers = GoodnessWeigher
enabled_backends = ontap-iscsi-1,ontap-iscsi-2
.
.
[ontap-iscsi-1]
volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
netapp_login = admin
netapp_password = *********
volume_backend_name = ontap-iscsi
netapp_server_hostname = 192.168.0.101
netapp_server_port = 80
netapp_transport_type = http
netapp_vserver = svm1
netapp_storage_protocol = iscsi
netapp_storage_family = ontap_cluster
filter_function = "volume.size < 5"

[ontap-iscsi-2]
volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
netapp_login = admin
netapp_password = *********
volume_backend_name = ontap-iscsi
netapp_server_hostname = 192.168.0.102
netapp_server_port = 80
netapp_transport_type = http
netapp_vserver = svm2
netapp_storage_protocol = iscsi
netapp_storage_family = ontap_cluster
filter_function = "volume.size >= 5 and capabilities.netapp_aggregate_used_percent < 45"

This cinder.conf file will schedule the creation of volumes as follows:

Cinder volumes that are of size < 5GB will be placed on the ontap-iscsi-1 backend.
Cinder volumes that are of size >= 5GB will be placed on the ontap-iscsi-2 backend, if the aggregate has a usage percent of lesser than 45%. Otherwise, volume creation will fail.

Example2: Using Goodness Weigher

[default]
.
.
scheduler_default_filters = DriverFilter,AvailabilityFilter,CapabilityFilter,CapacityFilter
scheduler_default_weighers = GoodnessWeigher
enabled_backends = ontap-iscsi-1,ontap-iscsi-2
.
.
[ontap-iscsi-1]
volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
netapp_login = admin
netapp_password = *********
volume_backend_name = ontap-iscsi
netapp_server_hostname = 192.168.0.101
netapp_server_port = 80
netapp_transport_type = http
netapp_vserver = svm1
netapp_storage_protocol = iscsi
netapp_storage_family = ontap_cluster
goodness_function = "(capabilities.utilization < 60.0) ? 60 : 30"

[ontap-iscsi-2]
volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
netapp_login = admin
netapp_password = *********
volume_backend_name = ontap-iscsi
netapp_server_hostname = 192.168.0.102
netapp_server_port = 80
netapp_transport_type = http
netapp_vserver = svm2
netapp_storage_protocol = iscsi
netapp_storage_family = ontap_cluster
goodness_function = "(capabilities.utilization < 60.0) ? 75 : 25"

In this example, the goodness_function is set for the available backends. For every volume request, the goodness function is calculated and used as follows:

If the node utilization for both backends ontap-iscsi-1 and ontap-iscsi-2 are lesser than 60%, the goodness weigher is set to 60 and 75 respectively. ontap-iscsi-2 would be preferred for the Cinder volume.
If the node utilization for ontap-iscsi-1 is greater than 60% and is lesser than 60 % for ontap-iscsi-2, the good weigher is higher (75%) for ontap-iscsi-2 than for ontap-iscsi-1 (30%).
If both backends have node utilization greater than 60%, then ontap-iscsi-1 would be preferred as it has a higher goodness weigher value (30 over 25)

Example3: Using Driver Filter and Goodness Weigher

[default]
.
.
scheduler_default_filters = DriverFilter,AvailabilityFilter,CapabilityFilter
scheduler_default_weighers = GoodnessWeigher
enabled_backends = sf-1,sf-2
.
.
[sf-1]
volume_driver = cinder.volume.drivers.solidfire.SolidFireDriver
volume_backend_name = sf-backend
san_login = admin
san_ip = 192.168.0.101
san_password = *************
filter_function = "(stats.allocated_capacity_gb + volume.size) < (stats.total_capacity_gb * 10.0)"
goodness_function = "((stats.allocated_capacity_gb) / (stats.total_capacity_gb) * 100 < 60.0) ? 60 : 30"

[sf-2]
volume_driver = cinder.volume.drivers.solidfire.SolidFireDriver
volume_backend_name = sf-backend
san_login = admin
san_ip = 192.168.0.102
san_password = *************
goodness_function = "((capabilities.max_iops - capabilities.average_iops)/capabilities.max_iops) * 100"

This example shows how the filter_function and goodness_function can be combined together. The filter_function for sf-1 evaluates if creating a requested volume can be performed without exceeding 10 * total capacity.

If this check passes, the goodness_function is evaluated for both backends based on node utilization. The backend with the higher value is ultimately selected.
If the filter_function check fails, the Driver Filter returns only one host, sf-2. The Goodness Weigher is calculated and the Cinder volume is placed on sf-2.