Spring Batch Example -- Read a CSV file, process it, write the output to an XML file

In this Spring Batch tutorial, show you how to create a job, read a CSV file, process it, and write the output to an XML file

1.Project Directory

2. Project Dependencies

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd" >
 <modelVersion>4.0.0</modelVersion>
 <groupId>SpringBatchExample</groupId>
 <artifactId>SpringBatchExample</artifactId>
 <packaging>jar</packaging>
 <version>1.0-SNAPSHOT</version>
 <name>SpringBatchExample</name>
 <url>http://maven.apache.org</url>
 <properties>
  <jdk.version>1.6</jdk.version>
  <spring.version>3.2.2.RELEASE</spring.version>
  <spring.batch.version>2.2.0.RELEASE</spring.batch.version>
  <mysql.driver.version>5.1.25</mysql.driver.version>
  <junit.version>4.11</junit.version>
 </properties>
 <dependencies>
  <!-- Spring Core -->
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-core</artifactId>
   <version>${spring.version}</version>
  </dependency>
  <!-- Spring jdbc, for database -->
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-jdbc</artifactId>
   <version>${spring.version}</version>
  </dependency>
  <!-- Spring XML to/back object -->
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-oxm</artifactId>
   <version>${spring.version}</version>
  </dependency>
  <!-- MySQL database driver -->
  <dependency>
   <groupId>mysql</groupId>
   <artifactId>mysql-connector-java</artifactId>
   <version>${mysql.driver.version}</version>
  </dependency>
  <!-- Spring Batch dependencies -->
  <dependency>
   <groupId>org.springframework.batch</groupId>
   <artifactId>spring-batch-core</artifactId>
   <version>${spring.batch.version}</version>
  </dependency>
  <dependency>
   <groupId>org.springframework.batch</groupId>
   <artifactId>spring-batch-infrastructure</artifactId>
   <version>${spring.batch.version}</version>
  </dependency>
  <!-- Spring Batch unit test -->
  <dependency>
   <groupId>org.springframework.batch</groupId>
   <artifactId>spring-batch-test</artifactId>
   <version>${spring.batch.version}</version>
  </dependency>

  <!-- Junit -->
  <dependency>
   <groupId>junit</groupId>
   <artifactId>junit</artifactId>
   <version>${junit.version}</version>
   <scope>test</scope>
  </dependency>
 </dependencies>
 <build>
  <finalName>spring-batch</finalName>
  <plugins>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-eclipse-plugin</artifactId>
    <version>2.9</version>
    <configuration>
     <downloadSources>true</downloadSources>
     <downloadJavadocs>false</downloadJavadocs>
    </configuration>
   </plugin>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.3.2</version>
    <configuration>
     <source>${jdk.version}</source>
     <target>${jdk.version}</target>
    </configuration>
   </plugin>
  </plugins>
 </build>
</project>
3. Spring Batch Jobs

A CSV file

report.csv

1001 "Nanda" "Gopal" "100000001" "test1@abc.com"
1002 "Ram" "Gopal "100000002" "test2@abc.com"
1003 "Muni" "Gopal" "100000003" "test3@abc.com"

A Spring batch job, to read above csv file with FlatFileItemReader, process the data with itemProcessor and write it to an XML file with StaxEventItemWriter.

job-emp-report.xml

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/batch
  http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-3.2.xsd" >
 <import resource="../config/context.xml" />
 <import resource="../config/database.xml" />
 <bean id="report" class="com.nadhu.springbatch.model.Report" scope="prototype" />
 <bean id="itemProcessor" class="com.nadhu.springbatch.EmployeeProcessor" />
 <batch:job id="empReportJob">
  <batch:step id="step1">
   <batch:tasklet>
    <batch:chunk reader="cvsFileItemReader" writer="xmlItemWriter" processor="itemProcessor"
     commit-interval="10">
    </batch:chunk>
   </batch:tasklet>
  </batch:step>
 </batch:job>
 <bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
  <property name="resource" value="classpath:cvs/input/report.csv" />
  <property name="lineMapper">
   <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
    <property name="lineTokenizer">
     <bean
      class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
      <property name="names" value="id,firstname,lastname,telephone,email" />
     </bean>
    </property>
    <property name="fieldSetMapper">
        <bean class="com.nadhu.springbatch.ReportFieldSetMapper" />
    </property>
   </bean>
  </property>
 </bean>
 <bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
  <property name="resource" value="file:xml/outputs/report.xml" />
  <property name="marshaller" ref="reportMarshaller" />
  <property name="rootTagName" value="report" />
 </bean>
 <bean id="reportMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
  <property name="classesToBeBound">
   <list>
    <value>com.nadhu.springbatch.model.Report</value>
   </list>
  </property>
 </bean>
</beans>

Map CSV value to Report object and write it to XML file (via jaxb annotations).
4. Report.java

package com.nadhu.springbatch.model;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement(name = "record")
public class Report {
 private int id;
 private String firstname;
 private String lastname;
 private String telephone;
 private String email;
 @XmlAttribute(name = "id")
 public int getId() {
  return id;
 }
 public void setId(int id) {
  this.id = id;
 }
 @XmlElement(name = "firstname")
 public String getFirstname() {
  return firstname;
 }
 public void setFirstname(String firstname) {
  this.firstname = firstname;
 }
 @XmlElement(name = "lastname")
 public String getLastname() {
  return lastname;
 }
 public void setLastname(String lastname) {
  this.lastname = lastname;
 }
 @XmlElement(name = "telephone")
 public String getTelephone() {
  return telephone;
 }
 public void setTelephone(String telephone) {
  this.telephone = telephone;
 }
 @XmlElement(name = "email")
 public String getEmail() {
  return email;
 }
 public void setEmail(String email) {
  this.email = email;
 }
 @Override
 public String toString() {
  return "Report [id=" + id + ", firstname=" + firstname + ", lastname="
    + lastname + ", telephone=" + telephone + ", email=" + email
    + "]";
 }
}

5. ReportFieldSetMapper.java

package com.nadhu.springbatch;
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;
import com.nadhu.springbatch.model.Report;
public class ReportFieldSetMapper implements FieldSetMapper {

 @Override
 public Report mapFieldSet(FieldSet fieldSet) throws BindException {

  Report report = new Report();
  report.setId(fieldSet.readInt(0));
  report.setFirstname(fieldSet.readString(1));
  report.setLastname(fieldSet.readString(2));
  report.setTelephone(fieldSet.readString(3));
  report.setEmail(fieldSet.readString(4));
  return report;
 }
}

A EmployeeProcessor will be fired before itemWriter.

6. EmployeeProcessor.java

package com.nadhu.springbatch;
import org.springframework.batch.item.ItemProcessor;
import com.nadhu.springbatch.model.Report;
public class EmployeeProcessor implements ItemProcessor {
 @Override
 public Report process(Report item) throws Exception {

  System.out.println("Processing..." + item);
  return item;
 }
}

7. Spring context and database configuration.

context.xml

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-3.2.xsd" >
 <!-- stored job-meta in database -->
 <bean id="jobRepository"
  class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
  <property name="dataSource" ref="dataSource" />
  <property name="transactionManager" ref="transactionManager" />
  <property name="databaseType" value="mysql" />
 </bean>

 <bean id="transactionManager"
  class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
 
 <bean id="jobLauncher"
  class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
  <property name="jobRepository" ref="jobRepository" />
 </bean>
</beans>

database.xml

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:jdbc="http://www.springframework.org/schema/jdbc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
  http://www.springframework.org/schema/jdbc
  http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd" >
    <!-- connect to database -->
 <bean id="dataSource"
  class="org.springframework.jdbc.datasource.DriverManagerDataSource">
  <property name="driverClassName" value="com.mysql.jdbc.Driver" />
  <property name="url" value="jdbc:mysql://localhost:3306/nadhu" />
  <property name="username" value="root" />
  <property name="password" value="xyz" />
 </bean>
 <bean id="transactionManager"
  class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
  <!-- create job-meta tables automatically -->
 <jdbc:initialize-database data-source="dataSource">
  <jdbc:script location="org/springframework/batch/core/schema-drop-mysql.sql" />
  <jdbc:script location="org/springframework/batch/core/schema-mysql.sql" />
 </jdbc:initialize-database>
</beans>

8. Run It (App.java)

package com.nadhu.springbatch;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class App {

 public static void main(String[] args) {
  String[] springConfig  =
   {
    "spring/batch/jobs/job-emp-report.xml"
   };

  ApplicationContext context =
    new ClassPathXmlApplicationContext(springConfig);

  JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
  Job job = (Job) context.getBean("empReportJob");
  try {
   JobExecution execution = jobLauncher.run(job, new JobParameters());
   System.out.println("Exit Status : " + execution.getStatus());
  } catch (Exception e) {
   e.printStackTrace();
  }
  System.out.println("Done");
 }
}
9. Output

report.xml

<?xml version="1.0" encoding="UTF-8"?>
<report>
 <record id="1001">
  <email>test1@abc.com</email>
  <firstname>Nanda</firstname>
  <lastname>Gopal</lastname>
  <telephone>100000001</telephone>
 </record>
 <record id="1002">
  <email>test2@abc.com</email>
  <firstname>Ram</firstname>
  <lastname>Gopal</lastname>
  <telephone>100000002</telephone>
 </record>
 <record id="1003">
  <email>test3@abc.com</email>
  <firstname>Muni</firstname>
  <lastname>Gopal</lastname>
  <telephone>100000003</telephone>
 </record>
</report>

10. In console

 INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@10821d79: defining beans [jobRepository,transactionManager,jobLauncher,dataSource,org.springframework.jdbc.datasource.init.DataSourceInitializer#0,report,itemProcessor,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,step1,empReportJob,cvsFileItemReader,xmlItemWriter,reportMarshaller]; root of factory hierarchy
15 Oct, 2015 10:34:38 PM org.springframework.jdbc.datasource.DriverManagerDataSource setDriverClassName
INFO: Loaded JDBC driver: com.mysql.jdbc.Driver
15 Oct, 2015 10:34:38 PM org.springframework.batch.core.launch.support.SimpleJobLauncher afterPropertiesSet
INFO: No TaskExecutor has been set, defaulting to synchronous executor.
15 Oct, 2015 10:34:39 PM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
INFO: Executing SQL script from class path resource [org/springframework/batch/core/schema-drop-mysql.sql]
15 Oct, 2015 10:34:39 PM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
INFO: Done executing SQL script from class path resource [org/springframework/batch/core/schema-drop-mysql.sql] in 593 ms.
15 Oct, 2015 10:34:39 PM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
INFO: Executing SQL script from class path resource [org/springframework/batch/core/schema-mysql.sql]
15 Oct, 2015 10:34:41 PM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
INFO: Done executing SQL script from class path resource [org/springframework/batch/core/schema-mysql.sql] in 1485 ms.
15 Oct, 2015 10:34:41 PM org.springframework.oxm.jaxb.Jaxb2Marshaller createJaxbContextFromClasses
INFO: Creating JAXBContext with classes to be bound [class com.nadhu.springbatch.model.Report]
15 Oct, 2015 10:34:41 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run
INFO: Job: [FlowJob: [name=empReportJob]] launched with the following parameters: [{}]
15 Oct, 2015 10:34:42 PM org.springframework.batch.core.job.SimpleStepHandler handleStep
INFO: Executing step: [step1]
Processing...Report [id=1001, firstname=Nanda, lastname=Gopal, telephone=100000001, email=test1@abc.com]
Processing...Report [id=1002, firstname=Ram, lastname=Gopal, telephone=100000002, email=test2@abc.com]
Processing...Report [id=1003, firstname=Muni, lastname=Gopal, telephone=100000003, email=test3@abc.com]
15 Oct, 2015 10:34:42 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run
INFO: Job: [FlowJob: [name=empReportJob]] completed with the following parameters: [{}] and the following status: [COMPLETED]
Exit Status : COMPLETED
Done
--------------
Thank you !