The DataFilter Plug-In

nevisDataPorter uses DataFilters to construct the output objects based on source objects loaded by a DataSource. The output objects will later be processed by a DataSink. For each source object processed a DataFilter will be called twice. The first call is to create attributes for the result object. The second call is to create control values (see also Attribute and Control Values below. The DataFilter does not have to track whether it has to create value or control attributes. The nevisDataPorter does so and passes the corresponding filter roles and the object.

A DataFilter can control whether nevisDataPorter should continue or abort the processing of the current record by its return value.

Nested Inner Loop

A DataFilter may have a local DataSources defined in its configuration (see The DataSource Plug-In). If a local DataSource exists, nevisDataPorter starts an inner loop for each record from the outer DataSource. The following pseudo code illustrates the nested loop.

while(outer.hasMoreObject()) {
  outer.getNextObject();
  inner.reset()
  while(inner.hasMoreObject()) {
    inner.getNextObject();
    outer.filter();
    outer.store()
  }
}

Nested inner loops are an advanced feature which allows to aggregate data from different data sources. The data from the DataSource is transformed using an expression and put into a list.

Attribute and Control Values

As mentioned above, each DataFilter is called twice. First to create all the attributes for the output object, which will later be stored on the target system. The second call is to create control values. Control values can later be used by any following DataFilter in the chain or the DataSource for any purpose, but they will never be stored on the target system. Thus a DataFilter may have to provide a set of mapping rules. In the configuration of a DataFilter the list of attributes for the output object comes first, followed by the attribute definitions for the control object. Attributes are like configuration values, they may be single valued, lists or map based attributes (see Configuration Section for a detailed description). The following extract shows an example configuration of the ELMappingDataFilter.

<dataFilter type="ELMappingFilter">
  <!-- Attribute definitions for the output object -->
  <dp:attrVal name="cn" value="#{in.person.LOGINID}"/>
  <dp:attrVal name="givenName" value="#{in.person.FIRSTNAME}"/>
  <dp:attrVal name="sn" value="#{in.person.LASTNAME}"/>
  <dp:attrVal name="unicodePwd" value="#{ctx:random( 8, 12 )}"

  <!-- Attribute definitions for the control object -->
  <dp:ctlVal name="operation"
    value="#{in.person.NUM_APPL>0?'createOrUpdate':'update'
    }"/>
</dataFilter>