Difficult internal variables

Question

I am doing a data migration project in Talend, and for one of the tasks I need to process a large table with many columns and match the (old) data with a different value for the new model.
I have a unique mapping table with three columns.

Example:

Column name | Value old | Value new "col 1" 1 3 "col 1" 3 2 "col 2" 10 7 etc 

That way, I can refer to the "column name" with values ​​related to the column that I need to display.

Using the old value as a lookup table in the mapping table using the "column name": when the old value matches, it returns the new value.

I could do it manually for each column, but there are hundreds to mean that at least one hash is repeated for each column.

I don’t understand how to make it easier and only once for each column in the source table.

Right now I'm doing a HUGE tMap and using a lot of searches: one for each column.

Any ideas are welcome.

-

Some additional ideas that I had:
1) Is there a way to find out that LINE NAME connects two components? I could just reuse the same connections and filter on tMap more easily

Answer 1

I could solve this with tMemorizeRows, storing the entire mapping table in memory using tJavaFlex, and then reading them using the procedure.

I did this, remembered the whole table, I had only one table with three columns, the first indicated the type of display that I had to do, the second - the source code, and the third - the replacement code.

After storing the data, I used tJavaFlex to create three lists, one for each column, and then moved these lists to the global variable map.

Then I created a procedure that gets all three lists plus two codes, the first is the name of the mapping (filter of the first column), and the second is the source code (filter of the second column). Using both options, I could determine the position of the replacement code and return it.

Finally, when reading the main data stream using tMap, I created a variable for each individual value of the first column (that is, one for each type of display), and then called routine i, designed using the original code + display name.

The code I used in tMemorizeRows was:

START CODE

 java.util.Set<String> iLista; java.util.List<String> lLOV=new java.util.ArrayList<String>(); java.util.List<String> lS6=new java.util.ArrayList<String>(); java.util.List<String> lS8=new java.util.ArrayList<String>(); 

MAIN CODE

 lLOV.add(LOV_tMemorizeRows_1[icount]); lS6.add(S6_NAME_tMemorizeRows_1[icount]); lS8.add(S8_NAME_tMemorizeRows_1[icount]); 

END OF CODE

 globalMap.put("lLOV",lLOV); globalMap.put("lS6",lS6); globalMap.put("lS8",lS8); 

ROUTINE CODE

 /* * Toma los valores generados de la LOV y matchea con S8, requiere la ejecucion del job * Genericos\GeneradorLOV * * {talendTypes} String | String * * {Category} MigracionDatos * * {param} string(entrada.LOV) Identifica el tipo de LOV * * {param} string(entrada.S6_NAME) Indica el valor del campo en Siebel 6 * * {param} ((java.util.List<String>) globalMap.get("lLOV")) * * {param} ((java.util.List<String>) globalMap.get("lS6")) * * {param} ((java.util.List<String>) globalMap.get("lS8")) * */ public static String calculaLOV(String CampoLOV, String CampoS6, java.util.List<String> listaLOV, java.util.List<String> listaS6,java.util.List<String> listaS8 ) { /* * java.util.List<String> listaLOV = ( java.util.List<String>) globalMap.get("lLOV"); * java.util.List<String> listaS6 = ( java.util.List<String>) globalMap.get("lS6"); * java.util.List<String> listaS8 = ( java.util.List<String>) globalMap.get("lS8"); */ String C1 = CampoLOV; String C2 = CampoS6; int posicionC1 = listaLOV.indexOf(C1); // encontró el LOV if(posicionC1 >= 0) { java.util.List<String> listaS6_Iterada = new java.util.ArrayList<String>(); // Genera la lista intermedia con los valores for (int contador = posicionC1; contador < listaLOV.size() ; contador++) { listaS6_Iterada.add(listaS6.get(contador)); if(!listaLOV.get(contador).toString().equals(C1)) {break;} } int posicionC2 = listaS6_Iterada.indexOf(C2); if(posicionC2 >= 0) { int posicionFinal = posicionC1 + posicionC2; return listaS8.get(posicionFinal); } else { return ""; } } else { return ""; } }