Create the Template

The next step is to create a template yaml file with the definitions to deploy the WordPress stack. This stack will launch 3 servers: HaProxy Load Balancer, WordPress Application server and a MySQL database server.

Refer to the Template Syntax section in this document for reference.

Create a new template file

Using your preferred text editor create a new template yaml file. Add the following statements, feel free to substitute values as appropriate.

template_name: "WordPress Two Server Stack using Puppet"
template_author: Shaun Marshall
template_version: "1.00"
description: "Deploys a WordPress two server stack using Puppet"
long_description: "Deploys a stack with a HaProxy Load Balancer, a WordPress Application and a MySQL Database Server."

imports:
  - base_types.yaml                                                   # Import the DCM base type definitions that are referenced in this template

Define a custom node type

In this tutorial a custom base server node type will be created. The custom base server node type will be used to define the three server nodes in the node_templates section of the template. This will reduce the amount of definitions required. The three server nodes will inherit the properties of the custom base server node type.

Add the statements to the yaml file to define the custom base server node type named base_vm.

node_types:

  ##################################################################################################################################################
  # This defines a custom node type which will be referenced (inherited from) the three server nodes types below in this template
  ##################################################################################################################################################
  base_vm:
    derived_from: dcm.nodes.Server
    properties:                                                                # Retrieve these server launch properties from the "inputs"
      cloud: { get_input: [account_region_zone_selector, cloud] }              # Retrieve cloud from the AccountRegionSelector
      cloudAccountId: { get_input: [account_region_zone_selector, accountId] } # Retrieve the cloud account ID from the AccountRegionSelector
      region: { get_input: [account_region_zone_selector, region] }            # Retrieve region from the AccountRegionSelector
      zone: { get_input: [account_region_zone_selector, zone] }                # Retrieve zone from the AccountRegionSelector
      platform: { get_input: [product_selector, platform] }                    # Retrieve platform from the Product selector
      architecture: { get_input: [product_selector, architecture] }            # Retrieve architecture from the Product selector
      product: { get_input: [product_selector, product] }                      # Retrieve product from the Product selector
      image: { get_input: [product_selector, image] }                          # Retrieve machine from the Product selector
      serverProductId: { get_input: [product_selector, serverProductId] }      # Retrieve server product id from the Product selector

      startupScript:  |
        #!/bin/bash
        ##########################################################################################################################################
        # Install curl on Debian/Ubuntu if necessary
        ##########################################################################################################################################
        which apt-get && apt-get update -qq && (which curl || apt-get install curl -y)

        ##########################################################################################################################################
        # Install curl on RHEL/CentOS if necessary
        ##########################################################################################################################################
        which yum && yum -q makecache && (which curl || yum install curl -y)

        ##########################################################################################################################################
        # Install the Dell Cloud Manager agent, point it to the Dell Cloud Manager server and configure it to startup at boot time.
        ##########################################################################################################################################
        export DCM_URL=${dcm.callback.url}

        ##########################################################################################################################################
        # The -Z argument configures the Dell Cloud Manager agent and allows it to accept unknown certificates.
        # This is only recommended for testing and should not be done in a production environment.
        # The --install-extras parameter installs the Puppet agent and extra required libraries on the DCM Agent.
        ##########################################################################################################################################
        curl -L --retry 10 https://linux-stable-agent.enstratius.com/installer.sh | bash -s - --url $DCM_URL --on-boot --install-extras -Z

        ##########################################################################################################################################
        # Start the Dell Cloud Manager agent.
        ##########################################################################################################################################
        /etc/init.d/dcm-agent start

Define topology_template and inputs

topology_template:

  inputs:

Define AccountRegionSelector

The next step is to define a AccountRegionSelector to allow the Dell Cloud Manager console end user the ability to select the server product for launching the servers in the stack.

  ################################################################################################################################################
  # This defines the AccountRegionSelector which allows the user to select the Cloud, Region and Datacenter
  ################################################################################################################################################
  account_region_zone_selector:                                     # Define the section for the Cloud, Region and Datacenter selection boxes
    type: dcm.inputs.AccountRegionSelector                          # Input type is dcm.inputs.accountRegionSelector 
    properties:
      regions:                                                      # Define the Cloud and Regions
        "Amazon":                                                   # Amazon
          "us-east-1": [ ]                                          # All data centers for the us-east-1 region
          "us-west-1": ["us-west-1a", "us-west-1c"]
          "us-west-2": ["us-west-2a", "us-west-2b", "us-west-2c"]
          "eu-west-1": ["eu-west-1a", "eu-west-1c"]

Define Product Selector

The next step is to define a Product selector to allow the Dell Cloud Manager console end user the ability to select the server product for launching the server(s) in the stack. This example will use the Amazon cloud and a subset of the regions and server products. An Ubuntu machine image will be defined in the template to be used for creating the servers.

Note

If you do not have access to the Amazon cloud or wish to use a different cloud provider’s cloud that Dell Cloud Manager supports, you will need to make the necessary changes below for the cloud, region(s), image(s) and products.

Add the statements to the yaml file to define the inputs product_selector. The inputs: statement must be indented inside the topology_template: statement. Refer to the Product Selector section in this document for reference.

    ################################################################################################################################################
    # This defines the Product selector which allows the user to select the server product size
    ################################################################################################################################################        
    product_selector:                                                 # Define the product selector so the user can select the cloud and region
      type: dcm.inputs.Product                                        # Input type is dcm.inputs.Product
      properties:
        accountRegionSelector: account_region_zone_selector           # This connects the AccountRegionSelector to the Product selector
        platform: UNIX                                                # Virtual machine images are Linux
        architecture: I64                                             # 64 bit images
        productMappings:
          "Amazon":                                                   # Amazon cloud
            "us-east-1":                                              # us-east-1 Region
              image: "ami-c4edc0d3"                                   # Machine image identifier for an Ubuntu image in this region
              products: ['t1.micro', 'm1.small', 'm1.medium']         # The virtual machine product sizes for this region
            "us-west-1":                                              # us-west-1 Region
              image: "ami-e7035687"                                   # Machine image identifier for an Ubuntu image in this region
              products: ['t1.micro', 'm1.small', 'm1.medium']         # The virtual machine product sizes for this region
            "us-west-2":                                              # us-west-2 Region
              image: "ami-578c2f37"                                   # Machine image identifier for an Ubuntu image in this region
              products: ['t1.micro', 'm1.small', 'm1.medium']         # The virtual machine product sizes for this region
            "eu-west-1":                                              # eu-west-1 Region
              image: "ami-e6a1f795"                                   # Machine image identifier for an Ubuntu image in this region
              products: ['t1.micro', 'm1.small', 'm1.medium']         # The virtual machine product sizes for this region

Add the statements below which will display output information on the running stack in the Stack Overview page. The IP address and web URL of the HaProxy Load Balancer will be displayed.

Define outputs

In this tutorial 6 outputs will be defined and displayed on the Stack details page.

  • IP address of the WordPress Application server
  • Web URL of the WordPress Web Application server
  • The user id used to login to the WordPress Application
  • The password used to login to the WordPress Web Application
  • Web URL of the HaProxy load balancer statistics page
  • The user id used to login to the HaProxy load balancer statistics page
  • The password used to login to the HaProxy load balancer statistics page

Add the statements to the yaml file to define the outputs. The outputs: statement must be indented inside the topology_template: statement and aligned with the inputs: statement. Refer to the Template Outputs section in this document for reference.

######################################################################################################################################################
# This defines three outputs to appear on the DCM console Stack overview page (IP address and URL of the WordPress Application, and HaProxy stats URL)
######################################################################################################################################################
outputs:                                                        # Define outputs
  application_group:                                            # Create a Group
    type: dcm.outputs.DisplayGroup                              # It's a displayGroup
    properties:
      displayName: "WordPress Application Information"          # Set the display name for the group containing the outputs which appears on the Stack Overview page

  vm_ip:                                                        # Define an output for the IP address of the launched server
    type: string                                                # It's a string output
    description: IP of the server                               # Set the description for the string outputs
    value: {get_attribute: [lb_vm, publicIpAddress]}            # Set the value to the public IP address of the server running in the stack
    properties:
      displayName: IP address                                   # Set the display name (label) for the string output
      displayGroup: application_group                           # Place this output in the displayGroup named application_group

  link:                                                         # Define another output for the URL of the launched server
    type: dcm.outputs.Uri                                       # It's a URI output
    value: http://0.0.0.0/index.php                             # Set the initial value
    properties:
      host: {get_attribute: [lb_vm, publicIpAddress]}           # Set the hostname/ipaddress of the URI to the public IP address of the server running in the stack
      displayName: WordPress Load Balancer URL                  # Set the display name (label) for the URI output
      displayGroup: application_group                           # Place this output in the displayGroup named application_group

  haproxy_group:                                                # Create a Group
    type: dcm.outputs.DisplayGroup                              # It's a displayGroup
    properties:
      displayName: "HaProxy Statistics"                         # Set the display name for the group containing the outputs which appears on the Stack Overview page

  haproxy_link:                                                 # Define another output for the URL of the HaProxy stats
    type: dcm.outputs.Uri                                       # It's a URI output
    value: http://0.0.0.0:1936/                                 # Set the initial value
    properties:
      host: {get_attribute: [lb_vm, publicIpAddress]}           # Set the hostname/ipaddress of the URI to the public IP address of the HaProxy server
      displayName: Load Balancer Stats URL                      # Set the display name (label) for the URI output
      displayGroup: haproxy_group

  haproxy_userid:                                               # Define another output for the URL of the HaProxy stats userid
    type: string                                                # It's a string output
    value: admin                                                # Set the initial value
    properties:
      displayName: Load Balancer Stats User Id                  # Set the display name (label) for the URI output
      displayGroup: haproxy_group

  haproxy_password:                                             # Define another output for the URL of the HaProxy stats password
    type: password                                              # It's a password output
    value: "Password12"                                         # Set the initial value
    properties:
      displayName: Load Balancer Stats Password                 # Set the display name (label) for the URI output
      displayGroup: haproxy_group

Define the node templates

The next step is to define 7 node_templates.

  1. A node_template to create a server for the the HaProxy Load Balancer.
  2. A node_template to create a server for the the WordPress Application server.
  3. A node_template to create a server for the MySQL Server.
  4. A node_template to install the MySQL Server using Chef.
  5. A node_template to install the WordPress Application using Chef.
  6. A node_template to install the HaProxy Load Balancer using Chef.
  7. A node_template to create a firewall used by the three servers.

Define the servers

Add the statements to define the server. In this example the Load Balancer server name will be hardcoded to be libvm, the WordPress application server name will be webvm and the SQL Server name will be dbvm. The remaining server properties will be retrieved from the corresponding properties derived from the product_selector and the Dell Cloud Manager console end user’s Launch Blueprint selections. Refer to the Server node section in this document for reference.

node_templates:

  ##################################################################################################################################################
  # This node_template defines a virtual machine named "lib_vm" which will host the HaProxy Load Balancer
  ##################################################################################################################################################
  lb_vm:
    type: base_vm                                               # Define load balancer and inherit the properties from the custom base server node base_vm
    properties:
      name: lbvm
    requirements:                                               # This virtual machine node has a requirement on the firewall
      - firewall: vm_firewall_rules
        relationship_type: tosca.relationships.DependsOn

  ##################################################################################################################################################
  # This node_template defines a virtual machine named "web_vm" which will host the WordPress Apache Web Server
  ##################################################################################################################################################
  web_vm:
    type: base_vm                                               # Define web server and inherit the properties from the custom base server node base_vm
    properties:
      name: webvm
    requirements:                                               # This virtual machine node has a requirement on the firewall
      - firewall: vm_firewall_rules
        relationship_type: tosca.relationships.DependsOn

  ##################################################################################################################################################
  # This node_template defines a virtual machine named "db_vm" which will host the MySQL database server
  ##################################################################################################################################################
  db_vm:
    type: base_vm                                               # Define db server and inherit the properties from the custom base server node base_vm
    properties:
      name:  dbvm
    requirements:                                               # This virtual machine node has a requirement on the firewall
      - firewall: vm_firewall_rules
        relationship_type: tosca.relationships.DependsOn

Define Puppet for HaProxy

Add the statements to define the Puppet node to install the WordPress application.

Attention

The puppetMaster: value must exactly match the name of the Puppet Account you configured in Dell Cloud Manager.

Refer to the Puppet section in this document for reference.

###################################################################################################################################################
# This node_template will result in Puppet installing the HaProxy server on the launched virtual machine named "lb_vm"
###################################################################################################################################################
puppet_haproxy:
  type: dcm.nodes.Puppet
  properties:
    puppetMaster: ACME Puppet
    classes: ['stack_wordpress::lb']
  requirements:
    - host: lb_vm

Define Puppet for WordPress

Add the statements to define the Puppet node to install the WordPress application.

Attention

The puppetMaster: value must exactly match the name of the Puppet Account you configured in Dell Cloud Manager.

Refer to the Puppet node section in this document for reference.

###################################################################################################################################################
# This node_template will result in Puppet installing the WordPress Web Server application on the launched virtual machine named "web_vm"
###################################################################################################################################################
puppet_wordpress:
  type: dcm.nodes.Puppet
  properties:
    puppetMaster: ACME Puppet
    classes: ['stack_wordpress::app']
    parameters:
      install_dir: "/opt/wp"
      db_name: "wordpress"
      db_pass: "averystrongpass"
      db_user: "root"
  requirements:
    - host: web_vm

Define Puppet for MySQL Server

Add the statements to define the Puppet node to install the MySQL database server.

Attention

The puppetMaster: value must exactly match the name of the Puppet Account you configured in Dell Cloud Manager.

Refer to the Puppet section in this document for reference.

###################################################################################################################################################
# This node_template will result in Puppet installing the MySQL database server on the launched virtual machine named "db_vm"
###################################################################################################################################################
puppet_mysql:
  type: dcm.nodes.Puppet
  properties:
    puppetMaster: ACME Puppet
    classes: ['stack_wordpress::db']
    parameters:
      mysql_root_pass: "averystrongpass"
  requirements:
    - host: db_vm

Define the Firewall

Add the statements to define the firewall vm_firewall_rules. Define rules to open HTTP port 80, HTTPS port 443, MySQL port 3306 and HaProxy stats port 196.

Refer to the FirewallGroup node section in this document for reference.

###################################################################################################################################################
# This node_template defines a firewall which opens the HTTP port 80, HTTPS port 443, MySQL port 3306 and HaProxy stats port 1936
###################################################################################################################################################
vm_firewall_rules:
  type: dcm.nodes.FirewallGroup                                              # This is a firewall
  properties:                                                                # Retrieve the cloud properties from the "inputs"
    name: "acme-fw-wordpress-stack"                                          # Set the firewall name
    cloud: { get_input: [account_region_zone_selector, cloud] }              # Retrieve the cloud from the AccountRegionSelector
    cloudAccountId: { get_input: [account_region_zone_selector, accountId] } # Retrieve the cloud account ID from the AccountRegionSelector
    region: { get_input: [account_region_zone_selector, region] }            # Retrieve the region from the AccountRegionSelector
    zone: { get_input: [account_region_zone_selector, zone] }                # Retrieve the zone from the AccountRegionSelector
    rules:
      - remote_ip_prefix: 0.0.0.0/0
        port: 80                                                             # Define a rule to open port 80  (HTTP)
      - remote_ip_prefix: 0.0.0.0/0
        port: 443                                                            # Define a rule to open port 443  (HTTPS)
      - remote_ip_prefix: 0.0.0.0/0
        port: 3306                                                           # Define a rule to open port 3306 (MYSQL)
      - remote_ip_prefix: 0.0.0.0/0
        port: 1936                                                           # Define a rule to open port 1936 (HAProxy Stats)

Define Auto Scaling Policy

In this tutorial 2 auto scaling policies will be defined to auto scale the WordPress Application Server. Refer to the Auto scaling section in this document for reference.

  • A scale up will be performed when the last 3 consecutive periods of cpu idle time are less than 20%.
  • A scale down will be performed when the last 2 consecutive periods of cpu idle time are greate than 80%.
groups:
  scaling_group_1:
    members: [web_vm]                      # The node_template with the statement named "web_vm" is a member of this group
    properties:
    instances: 2                           # The initial number of servers to start when the stack is started
    minInstances: 1                        # The minimum number of servers
    maxInstances: 10                       # The maximum number of servers
    coolDown: 300                          # The number of seconds to wait before performing a auto scale or auto repair operation.

  policies:
  # This scale up policy will perform a scale up when the last 3 periods of idle time reported are < 20%
    scale_up_on_cpu:
      type: dcm.policy.types.BasicPolicy
      actions: [scale_up]                  # The scale_up action is defined in the actions: section of the template
      measurement: cpu_idle_time           # The cpu_idle_time label is defined in the measurements: section of the template
      criterion: less_than_20              # The less_than_20 is a label defined in the criteria: section of the template

  # This scale down policy will perform a scale down when the last 2 periods of idle time reported are > 80%
    scale_down_on_cpu:
      type: dcm.policy.types.BasicPolicy
      actions: [scale_down]                # The scale_down action is defined in the actions: section of the template
      measurement: cpu_idle_time           # The cpu_idle_time label is defined in the measurements: section of the template
      criterion: more_than_80              # The more_than_80 is a label defined in the criteria: section of the template

  # Actions define the details of the various actions that could be taken.
  actions:
    scale_up:                              # The scale_up action label is referenced in the policy in the actions[] statement
      type: dcm.policy.action.ScaleUpGroup
      properties:
        instances: 1                       # The number of resources to scale up on a scale_up action
        changeType: ADD                    # Add resource(s) on a scale_up action
        recordTask: RESOURCES_CHANGE       # Check to see if the scale up action would actually create a new resource,
                                           # or a scale-down operation would actually destroy a resource before creating a task to perform the action.

    scale_down:                            # The scale_down action label is referenced in the policy in the actions[] statement
      type: dcm.policy.action.ScaleDownGroup
      properties:
        instances: 1                       # The number of resources to scale down on a scale_down action
        changeType: REMOVE                 # Remove resource(s) on a scale_down action
        recordTask: ALWAYS                 # Always attempt to perform a scale up or scale down action when the policy determines the actions should occur.
                                           # Do not check the minInstances and maxInstances limits beforehand.

  # Measurements define the "measurements" used to determine when to perform the actions
  measurements:
    cpu_idle_time:                         # The cpu_idle_time measurements label is referenced in the policy in a measurements: statement
    # The Dell Cloud Manager agent will collect and store the last 15 samples of cpu %idle time measured in 30-second intervals
      type: dcm.policy.measurement.CpuIdle
      properties:
        period: 30                         # Take a measurement every 30 seconds
        count: 15                          # Take 15 measurements

  # Criteria specify the "criteria" which is used along with the measurements to determine when to perform the actions
  criteria:
    less_than_20:                          # The less_than_20 criteria label is referenced in the policy in a criterion: statement
      type: dcm.policy.criteria.SeriesLessThan
      properties:
        count: 3                           # 3 consecutive periods
        threshold: 20                      # Threshold is 20%

    more_than_80:                          # The more_than_80 criteria label is referenced in the policy in a criterion: statement
      type: dcm.policy.criteria.SeriesMoreThan
      properties:
        count: 2                           # 2 consecutive periods
        threshold: 80                      # Threshold is 80%

Define Auto Healing Policy

Now lets add an auto healing policy to the template.

Add the following statements inside the policy: section and align them with the two scaling policies.

repair_on_status:
  type: dcm.policy.types.BasicPolicy
  measurement: cloud_reported_status
  criterion: check_fails
  actions: [repair]

Add the following statements inside the actions: section and align it with the two scaling actions.

repair:                                    # This auto healing action will terminate a degraded resource and then create a new resource
  type: dcm.policy.action.ReplaceResource

Add the following statements inside the measurements: section and align them with the two scaling measurements.

cloud_reported_status:
  type: dcm.policy.measurement.ResourceActive

resource_status:
  type: dcm.policy.measurement.ResourceStatus

Lastly add the following statements inside the criteria: section and align them with the two scaling criteria.

check_fails:                               # This is the criteria for auto healing
  type: dcm.policy.criteria.False          # "False" tells the policy to perform the action in the policy if the measurement result matches "False".