The context is the message: Cascading Configuration Pattern

Synopsis

Load configuration files in cascade to reduce duplicate data.

Keywords: SCM, CM, Properties, Groovy, Config, CSS

Content: Context, Forces, Solution, Consequences, Implementation, Code Example, Related Patterns, Related Intellectual Property, Further Reading

Context

A business system uses property (configuration) files to configure particular environments or subsystems. Many environments share the same properties and values, however, some are different and crucial. To avoid missing any properties, all the properties are duplicated in each file, and required differences are changed appropriately.

For example, if there are 100 properties required and there are 23 possible environments, that is 2300 lines of source to manage. If there are any duplicated properties that do not vary between environments, then there is an opportunity to simplify the configuration system. In this case, if we make only a "root" file have the full set of properties, the total size is given by: T = L+npL; where T is total size, n is number of files, and p is percent of each file that is overridden. Here the value is 560=100+23*.2*100). The reduction over the duplicated data is 75%.

Forces

There are many issues with having the same properties in multiple locations. One obvious disadvantage, is adding a new property would require changing multiple files. Also the files are larger and not really cohesive. Tracking errors is also complicated, especially run-time errors due to configuration settings. Any solution should not add its own complexity. Thus, we should not require new types of configuration files and requirements.

Even when configuration could be isolated or systems modularized, there may be a need to have duplicated properties or reassignment to satisfy software development life cycle (SDLC). A system will be different in dev, test, and production environments.

Solution

A hierarchy of property sources and a system that can load each source and override properties at a prior level will provide a partial solution. This is the approach already taken in many systems. For example, in software applications and servers, the final properties used are composed of those taken from various standard locations in a particular Operating System host. In Windows for example, the Environment Variables are composed of those found in System and User name space. In software tools, such as Mercurial DVCS, there is a command to show the results of the final configuration after all configuration sources are loaded: showconfig show combined config settings from all hgrc files

Many build systems such as Ant and Maven, allow and expect the cascading of property files. Note that in Ant, the inverse policy is used, the first property assignment wins.

And, of course, this cascading is seen in the Cascading Style Sheet (CSS) technology.

Consequences

There is now the requirement that the cascade order is well known, non-variable, and robust. Any intervening file must be loaded, else the system may destabilize. Adding a new property or changing a property will require extra care since an environment or subsystem may be effected if the file contents are not properly designed beforehand.

To provide management and debugging support the implementation should provide traceability of configuration. Thus, during use, the system must report (log) what was overridden, not changed, what is missing, etc.

Implementation

Using this pattern is very easy. One just has to determine how the system handles reassignment in the configuration loader subsystem. If it allows resetting then the file hierarchy is from global to specific.

Where complexity may come is when one wants to satisfy more sophisticated requirements. For example, a property may need to be appended. In OS paths, for instance, paths are combined to create the final path. Thus, the operations on properties could be a combination of:

create: initial assignment of a property
update: a new value is assigned
delete: the property is removed
append: append to existing value
merge: use a pattern to merge a new value
locked: allow no change
fail on replace: if a value already assigned throw an exception.

Code Example

In Java the java.util.Properties class will overwrite an existing property with the same key value. Thus, to cascade properties, one just reuses the same instantiated Properties Object via the various load(...) methods.

Shown in listing one is a simple implementation written in Groovy, a dynamic language on the JVM. The PropCascade class extends Properties and adds methods to load a list of sources. The first source in the list is the root of the "cascade". In addition, a few methods explicitly specify the root source. From a traceability point of view, reusing java.util.Properties class may not be optimal. It gives no access to low level capture of the actual "put" action. Even with the use AOP, there is no joinpoint available to add it.

[sourcecode language="groovy"]
/**
* File: CascadedProperties.groovy
* Date: 23OCT10T18:13-05
* Author: JBetancourt
*/

import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Properties;

/**
* An extension of Properties that adds Convenience
* methods to load lists of sources.
*
* @author jbetancourt
*/
class CascadedProperties extends Properties {
 //private Properties rootProperties = new Properties();
 //private boolean firstWins = true;
 //private boolean failOnDuplicate = false;
 //private boolean isTrace = false;

 /**
  * Load a list of properties sources.
  * @param list
  */
 public void load(List list) throws IOException, IllegalArgumentException {
  list.each {
   load(it)
  }
 }

 /**
  * Explicit file path is specified.
  * @param path
  */
 public void load(String path) throws IOException, IllegalArgumentException {
  load(new File(path).newInputStream());
 }

 /**
  * A load method that explicitly specifies the "default" source in
  * the cascade order.
  *
  * @param inStream
  * @param list
  */
 public void load(InputStream inStream, List list) throws IOException, IllegalArgumentException {
  load inStream
  load list
 }

 /**
  * A load method that explicitly specifies the "default" source in
  * the cascade order.
  *
  * @param reader
  * @param list
  */
 public void load(Reader reader, List list) throws IOException, IllegalArgumentException {
  load reader
  load list
 }

 /**
  * A load method that explicitly specifies the "default" source in
  * the cascade order.
  *
  * @param path
  * @param list
  */
 public void load(String path, List list) throws IOException, IllegalArgumentException {
  load path
  load list
 }

} // end of CascadedProperties
[/sourcecode]

In listing two, the JUnit test class is shown.

[sourcecode language="groovy"]
/**
* File: CascadedPropertiesTest.groovy
* Date: 23OCT10T18:13-05
* Author: JBetancourt
*/

import java.io.File;
import java.io.FileInputStream;
import java.util.List;

import org.junit.Before;
import org.junit.Test;

import groovy.util.GroovyTestCase;

/**
* Test the {@link CascadedProperties} class.
*/
class CascadedPropertiesTest extends GroovyTestCase{
 private CascadedProperties cp;

 /** excuted before each test method run */
 public void setUp() throws Exception {
  cp = new CascadedProperties();
 }

 public void testloadListPaths() throws Exception {
  List list = new ArrayList();
  list.add path1
  list.add path2

  cp.load(list);

  assertEquals("v2",cp.get("k1"));
 }

 public void testloadListReaders() throws Exception {
  List list = new ArrayList();
  list.add reader1
  list.add reader2

  cp.load(list);

  assertEquals("v2",cp.get("k1"));
 }

 public void testloadListStreams() throws Exception {
  List list = new ArrayList();
  list.add iStream1
  list.add iStream2

  cp.load(list);

  assertEquals("v2",cp.get("k1"));
 }

 public void testloadStreamAndListStreams() throws Exception {
  List list = new ArrayList();
  list.add iStream2
  list.add iStream3

  cp.load(iStream1,list);

  assertEquals("v3",cp.get("k1"));
 }

 public void testloadPathAndListStreams() throws Exception {
  List list = new ArrayList();
  list.add iStream2
  list.add iStream3

  cp.load("data\\file1.properties",list);

  assertEquals("v3",cp.get("k1"));
 }

 public void testloadReaderAndListStreams() throws Exception {
  List list = new ArrayList();
  list.add reader2
  list.add reader3

  cp.load(reader1,list);

  assertEquals("v3",cp.get("k1"));
 }

 public void testPutAgain() {
  cp.put("k1", "v1")
  cp.put("k1", "v2")
  assertEquals(cp.get("k1"), "v2");
 }

 public void testLoadOneFilePath() throws Exception {
  cp.load("data\\file1.properties");
  assertEquals("v1",cp.get("k1"));
 }

 public void testLoadTwoFiles() throws Exception {
  cp.load(iStream1)
  cp.load(iStream2)

  assertEquals("v2",cp.get("k1"));
 }

 //
 // class fields
 //
 String path1 = "data\\file1.properties"
 String path2 = "data\\file2.properties"
 String path3 = "data\\file3.properties"
 File file1 = new File(path1)
 File file2 = new File(path2)
 File file3 = new File(path3)

 InputStream iStream1 = file1.newInputStream()
 InputStream iStream2 = file2.newInputStream()
 InputStream iStream3 = file3.newInputStream()
 Reader reader1 = file1.newReader()
 Reader reader2 = file2.newReader()
 Reader reader3 = file3.newReader()

}
[/sourcecode]

Related Patterns

Related Intellectual Property

"Cascading configuration using one or more configuration trees", U.S. Patent number 7760746, 30Nov2004, http://patft.uspto.gov/netacgi/nph-Parser?Sect2=PTO1&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN%2F7760746

The context is the message

Wednesday, November 03, 2010

Cascading Configuration Pattern