Spring -Batch Multi-line Writer Writer Element with Variable Number of Lines per Write

Question

I have the following requirement, but I cannot decide on the approach:

I need to write data to a fixed format file, where each record spans several lines, as shown below:

000120992599999990000000000000009291100000000000000000000000010000 000000000000000000000006050052570009700000050000990494920000111100 ABCDE:WXYZ 0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 000000000000000000000006050052730005700001100000090494920000221200 ABCDE:WXYZ 0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 000000000000000000000006050113110009700000000000000494920000311100 ABCDE:WXYZ 0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 000012099259999999000000000000000929110000000000000000000000001000 

This is one entry from the above example:

 000000000000000000000006050052570009700000050000990494920000111100 ABCDE:WXYZ 0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 descriptiongoesheredescriptiongoesheredescriptiongoesher0200 

The first and last line is the header and footer, respectively. The first line of each entry contains several details. The second line contains some other details with spaces.

I have a long description field that I need to divide by 56 characters and add those that have a third line entry in front.

So, in some records this can be only one line, and in some there can be three lines.

I need guidance on designing my itemwriter in the above scenario.

Nick

Answer 1

in the official spring-batch-samples there is an example of multi-line recording, multiline.xml search and MultiLineTradeItemWriter

this is basically the usual principle of delegation, you only need the desired domain object with a valid list of those intermediate lines 1..n

  public class MultiLineTradeItemWriter implements ItemWriter<Trade>, ItemStream { private FlatFileItemWriter<String> delegate; public void write(List<? extends Trade> items) throws Exception { List<String> lines = new ArrayList<String>(); for (Trade t : items) { lines.add("BEGIN"); lines.add("INFO," + t.getIsin() + "," + t.getCustomer()); lines.add("AMNT," + t.getQuantity() + "," + t.getPrice()); lines.add("END"); } this.delegate.write(lines); } } 

Answer 2

you can do it in CustomExtractor.

for example:

  public Object[] extract(T item) { List<Object> values = new ArrayList<Object>(); BeanWrapper bw = new BeanWrapperImpl(item); for (String propertyName : this.names) { values.add( bw.getPropertyValue(propertyName) + "\r\n"); } return values.toArray(); } 

Answer 3

I had a similar problem with writing multiple rows to the database. Since the job step creates a list of your items, how can you return a list from a process to a writer? This will create a list of lists, and the doWrite method in Writer is not configured to handle this script.

I am using 1.2.0 spring-boot-starter-parent (which gives me spring -core 4.1.3) along with sleeping (4.3.7). In my case, I have a “receivable”, and for every 1 receivable I read from the csv file, I may need to update or insert many receivables into my table. I also use HibernateItemWriter.

My solution was to extend HibernateItemWriter. The sessionFactory field is private, so I pass it through the constructor, as well as using setter (to accommodate existing code). Thus, my source code (which works just fine for one “receivable”) initially looked like this:

  @Bean public ItemWriter<Receivable> recivableWriter() { HibernateItemWriter<Receivable> hibernateItemWriter = new HibernateItemWriter<Receivable>(){{ setSessionFactory(sessionFactory); }}; return hibernateItemWriter; } 

After expanding HibernateItemWriter and changing my processor to return the list, my code changed to:

  @Bean public ItemWriter<List<Receivable>> receivableWriter() { HibernateListItemWriter<List<Receivable>> hibernateItemWriter = new HibernateListItemWriter<List<Receivable>>(this.sessionFactory){{ setSessionFactory(sessionFactory); }}; return hibernateItemWriter; } 

And my extended class looks like this (I can clear it, but this is my initial pass. I also want sessionFactory and clearSession fields not to be private)

 package com.work; import java.util.ArrayList; import java.util.List; import org.hibernate.SessionFactory; public class HibernateListItemWriter<Receivable> extends org.springframework.batch.item.database.HibernateItemWriter<Receivable> { public HibernateListItemWriter(SessionFactory sessionFactory) { super(); this.sessionFactory = sessionFactory; } private SessionFactory sessionFactory; private boolean clearSession = true; @Override public void write(List items) { List<Receivable> unwrappedItems = new ArrayList<Receivable>(); List<List<Receivable>> wrappedItems = (List<List<Receivable>>)items; for (List<Receivable> singleList: wrappedItems) { unwrappedItems.addAll(singleList); } doWrite(sessionFactory, unwrappedItems); sessionFactory.getCurrentSession().flush(); if(clearSession) { sessionFactory.getCurrentSession().clear(); } } } 

A big scream at grepcode.com to make life easier.