Unorthodox Puppet Scripting - Creating Loops and Nested loops in Puppet

What is Puppet?

Puppet is automation software for IT system admins, consultants etc. It allows you to automate repetitive tasks such as the installation of applications and services, patch management, and deployments. but it has many bottlenecks when it comes to do some standalone deployments.

You can read more about puppet from here.

Why Unorthodox?

So why do I call this Unorthodox? Simply because puppet was not built to do deployments the way I'm trying to do. What I'm trying to achieve is, no Puppet Master, no modules in the /etc/puppet directory and no need run the scripts as the root user.

Does this make any sense? So if you are not sure what the heck I'm trying to do, following is a use case where I need to do something like above.

The Use Case :

I need to setup a application cluster which have multiple Manager/Worker nodes, and the user can parse puppet manifest the number of nodes that is required as a parameter. The setup process will include two repetitive tasks 

1. Copying Files
2. Changing Configurations

I want to do this in a single go, I don't want to re-run the scripts to match the number of nodes. So in-order to do the repetitive tasks I need loops, so one loop has to copy necessary files and create required number of nodes and the other nested loop should push all the configuration templates. 

Please note that I'm writing these scripts to be executed without the PUPPET MASTER. All the resources and classes will be defined inline so I don't have to include the resources as puppet modules. I believe you get the point here. The deployment will be done on a local machine and can be transferred to remote machines if required by simply copying the setup.

 Lets Get Started.

The first challenge is to get a loop working, puppet doesn't have native support for loops so you will have to follow a work around in order to create a loop.

So lets start!!!

How do we create a loop!!! Hmm this took lots of Googling and lot of trying-out :).

Following is a for loop where you can parse the loop starting value and the max count to the loop.
So lets create a puppet.pp manifest and include the following code within,

class loop_test {
  #Invoking the loop resource name is the loop starting point.
  loop{"1":
    count=>"5", # Loop ending index
  }

}

define loop($count) {

  if ($name > $count) {
    notice("Loop Iteration Finished!!!\n")
  }
  else
  {
    notice("The Iteration Number : $name \n")


    $next = $name + 1
    loop { $next:
      count => "${count}",
    }
  }
}

include loop_test

You can run the above manifest by "puppet apply puppet.pp" command.  VOILA, the output will look like the following. 

Notice: Scope(Loop[1]): The Iteration Number : 1 

Notice: Scope(Loop[2]): The Iteration Number : 2 

Notice: Scope(Loop[3]): The Iteration Number : 3 

Notice: Scope(Loop[4]): The Iteration Number : 4 

Notice: Scope(Loop[5]): The Iteration Number : 5 

Notice: Scope(Loop[6]): Loop Iteration Finished!!!

So there you go, you have a working "for loop" in puppet now. Now lets see how can we have a nested loop in this loop. There are two ways you can do this, one is simply iterate over an array or define another resource that you can recursively call.

Here I will iterate over an Array because I find it less complicated :) I have defined all the config changes (paths of .erb templates) in an array so I will simply iterate over the array after copying the necessary files.

The following code segment can be used to achieve this.

class loop_test {
  #Invoking the loop resource name is the loop starting point.
  loop{"1":
    count=>"5", # Loop ending index
  }

  $config_array = ["Value 01","Value 02","Value 03","Value 04","Value 05"]

}


define loop($count) {

  if ($name > $count) {
    notice("Loop Iteration Finished!!!\n")
  }
  else
  {
    notice("The Iteration Number : $name \n")

    # This is to avoid any resource name duplications
    $local_names = regsubst($loop_test::config_array, '$', "-$name")

    # Invoking the second iteration
    push_templates{$local_names:
      loop_index => $name,
    }

    $next = $name + 1
    loop { $next:
      count => "${count}",
    }
  }
}

define push_templates($loop_index) {

  $orig_name = regsubst($name, '-[0-9]+$', '') # Extract the Original name

  # Do something Here

  notice("The Loop Index : ${loop_index} \n The Array Content : $orig_name \n")
}

include loop_test

The output Will look like following after executing the above script.

Notice: Scope(Loop[1]): The Iteration Number : 1 

Notice: Scope(Push_templates[Value 01-1]): The Loop Index : 1 
 The Array Content : Value 01 

Notice: Scope(Push_templates[Value 02-1]): The Loop Index : 1 
 The Array Content : Value 02 

Notice: Scope(Push_templates[Value 03-1]): The Loop Index : 1 
 The Array Content : Value 03 

Notice: Scope(Push_templates[Value 04-1]): The Loop Index : 1 
 The Array Content : Value 04 

Notice: Scope(Push_templates[Value 05-1]): The Loop Index : 1 
 The Array Content : Value 05 

Notice: Scope(Loop[2]): The Iteration Number : 2 

Notice: Scope(Push_templates[Value 01-2]): The Loop Index : 2 
 The Array Content : Value 01 

Notice: Scope(Push_templates[Value 02-2]): The Loop Index : 2 
 The Array Content : Value 02 

Notice: Scope(Push_templates[Value 03-2]): The Loop Index : 2 
 The Array Content : Value 03 

Notice: Scope(Push_templates[Value 04-2]): The Loop Index : 2 
 The Array Content : Value 04 

Notice: Scope(Push_templates[Value 05-2]): The Loop Index : 2 
 The Array Content : Value 05 

Notice: Scope(Loop[3]): The Iteration Number : 3 

Notice: Scope(Push_templates[Value 01-3]): The Loop Index : 3 
 The Array Content : Value 01 

Notice: Scope(Push_templates[Value 02-3]): The Loop Index : 3 
 The Array Content : Value 02 

Notice: Scope(Push_templates[Value 03-3]): The Loop Index : 3 
 The Array Content : Value 03 

Notice: Scope(Push_templates[Value 04-3]): The Loop Index : 3 
 The Array Content : Value 04 

Notice: Scope(Push_templates[Value 05-3]): The Loop Index : 3 
 The Array Content : Value 05 

Notice: Scope(Loop[4]): The Iteration Number : 4 

Notice: Scope(Push_templates[Value 01-4]): The Loop Index : 4 
 The Array Content : Value 01 

Notice: Scope(Push_templates[Value 02-4]): The Loop Index : 4 
 The Array Content : Value 02 

Notice: Scope(Push_templates[Value 03-4]): The Loop Index : 4 
 The Array Content : Value 03 

Notice: Scope(Push_templates[Value 04-4]): The Loop Index : 4 
 The Array Content : Value 04 

Notice: Scope(Push_templates[Value 05-4]): The Loop Index : 4 
 The Array Content : Value 05 

Notice: Scope(Loop[5]): The Iteration Number : 5 

Notice: Scope(Push_templates[Value 01-5]): The Loop Index : 5 
 The Array Content : Value 01 

Notice: Scope(Push_templates[Value 02-5]): The Loop Index : 5 
 The Array Content : Value 02 

Notice: Scope(Push_templates[Value 03-5]): The Loop Index : 5 
 The Array Content : Value 03 

Notice: Scope(Push_templates[Value 04-5]): The Loop Index : 5 
 The Array Content : Value 04 

Notice: Scope(Push_templates[Value 05-5]): The Loop Index : 5 
 The Array Content : Value 05 

Notice: Scope(Loop[6]): Loop Iteration Finished!!!

Everything is done, and working as expected :)

I have written some scripts using above concepts to deploy WSO2 products to setup test environments. You can find puppet scripts I have written from the following Git Repository.

https://github.com/ycrnet/Puppet-Scripts

Please feel free to drop a comment if you have any queries!!

Thanks for Reading.