Spring Batch- Read a CSV file and write to an XML file

이번 예제에서는 FlatFileItemReader을 사용하는 CSV file을 읽기 위해서, StaxEventItemWrite XML file을 쓰기 위해서, 어떻게 Spring Batch를 사용하는지를 배울 것이다.

JobExecutionListener and itemProcessor  어법 또한 볼 수 있을 것이다

 

Following technologies being used:

• Spring Batch 3.0.1.RELEASE
• Spring core 4.0.6.RELEASE
• Spring oxm 4.0.6.RELEASE
• Joda Time 2.3
• JDK 1.6
• Eclipse JUNO Service Release 2

 

 

 

 

 

 Step 1: Create project directory structure

 

 

 

Step 2: Update pom.xml to include required dependencies

Pom.xml을 다음처럼 수정

 

 

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.websystique.springbatch</groupId>
	<artifactId>SpringBatchCsvToXml</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>SpringBatchCsvToXml</name>
	<url>http://maven.apache.org</url>

	<properties>
		<springframework.version>4.0.6.RELEASE</springframework.version>
		<springbatch.version>3.0.1.RELEASE</springbatch.version>
		<joda-time.version>2.3</joda-time.version>

		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${springframework.version}</version>
		</dependency>
		<!-- Object/XML 매핑지원(XML문서를 객체로 또는 객체를 XML로 변환 -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-oxm</artifactId>
			<version>${springframework.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${springbatch.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-infrastructure</artifactId>
			<version>${springbatch.version}</version>
		</dependency>
		
		<!-- java.util.Calendar 와 java.util.DateTime 이 사용하기 불편하고 버그가 많고 확장도 어려움
		Joda-Time 은 JDK 의 Calendar와 Date 를 대체하는 library.
		 -->
		<dependency>
			<groupId>joda-time</groupId>
			<artifactId>joda-time</artifactId>
			<version>${joda-time.version}</version>
		</dependency>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>3.8.1</version>
			<scope>test</scope>
		</dependency>
	</dependencies>
	<build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.2</version>
                    <configuration>
                        <source>1.6</source>
                        <target>1.6</target>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
	
</project>

 

여기서 date-time처리를 위해 joda-time api 사용할 것이다.

 

Step 3: Prepare the input flat file and corresponding domain object /mapped POJO

아래는 ‘|’로 분리되는 입력파일이다. 이 데이터는 우리가 XML format으로 전환할 것이다

Src/main/resources/ExamResult.txt

 

Brian Burlet   |   01/02/1985  |   76
Jimmy Snuka    |   01/02/1983  |   39
Renard konig   |   01/02/1970  |   61
Kevin Richard  |   01/02/2002  |   59
Rita Paul      |   01/02/1993  |   92
Han Yenn       |   01/02/1965  |   83

그리고 위 파일의 행에서 일치하는 필드와 POJO를 맵핑시킬 것이다.

 

package com.websystique.springbatch.model;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;

import org.joda.time.LocalDate;

@XmlRootElement(name = "ExamResult")
public class ExamResult {

	private String studentName;

	private LocalDate dob;

	private double percentage;

	@XmlElement(name = "studentName")
	public String getStudentName() {
		return studentName;
	}

	public void setStudentName(String studentName) {
		this.studentName = studentName;
	}

	@XmlElement(name = "dob")
	@XmlJavaTypeAdapter(type = LocalDate.class, value = com.websystique.springbatch.LocalDateAdapter.class)
	public LocalDate getDob() {
		return dob;
	}

	public void setDob(LocalDate dob) {
		this.dob = dob;
	}

	@XmlElement(name = "percentage")
	public double getPercentage() {
		return percentage;
	}

	public void setPercentage(double percentage) {
		this.percentage = percentage;
	}

	@Override
	public String toString() {
		return "ExamResult [studentName=" + studentName + ", dob=" + dob + ", percentage=" + percentage + "]";
	}

}

또한 the class 속성값들을 XML tags에 map시키기 위해 JAXB annotations 를 사용하고 있다

Joda-Time LocalDate API를 사용하기 떄문에, 어떻게 변환을 할 건지 JAXB에 전달해줘야 한다.

 

 

 

package com.websystique.springbatch;


import javax.xml.bind.annotation.adapters.XmlAdapter;

import org.joda.time.LocalDate;

public class LocalDateAdapter extends XmlAdapter
 {

	@Override
	public String marshal(LocalDate v) throws Exception {
		return v.toString();
	}

	@Override
	public LocalDate unmarshal(String v) throws Exception {
		return new LocalDate(v);
	}

}

 

 

Step 4: Create a FieldSetMapper

FieldSetMapper는 입력에 의한 각각의 필드를 domain 객체에 mapping 시켜주는 역할을 한다.

 

 

package com.websystique.springbatch;

import org.joda.time.LocalDate;
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;

import com.websystique.springbatch.model.ExamResult;

public class ExamResultFieldSetMapper implements FieldSetMapper
{

	@Override
	public ExamResult mapFieldSet(FieldSet fieldSet) throws BindException {
		 ExamResult result = new ExamResult();
	        result.setStudentName(fieldSet.readString(0));
	        result.setDob(new LocalDate(fieldSet.readDate(1,"dd/MM/yyyy")));
	        result.setPercentage(fieldSet.readDouble(2));
	        return result;

	}

}

 

 

Step 5: Create an ItemProcessor

ItemProcessor는 선택적이고, item을 읽은 후에 호출되어지고 item을 쓰기 전에 호출된다.

이것은 각각의 item의 business logic을 이행할 수 있는 기회를 준다.

예를들면, 60% 이하의 item은 걸러낼 수 있다. (즉 60이상의 item만 기록)

 

package com.websystique.springbatch;

import org.springframework.batch.item.ItemProcessor;

import com.websystique.springbatch.model.ExamResult;

public class ExamResultItemProcessor implements ItemProcessor
 {

	@Override
	public ExamResult process(ExamResult result) throws Exception {
		  System.out.println("Processing result :"+result);
		  
		  //60 이상만 리턴
		  if(result.getPercentage() < 60){
			  return null;
		  }
		  
		  return result;

	}

}

 

 

Step 6: Add a Job listener(JobExecutionListener)

Job listener은 선택적이며, job(작업) 시작되기 전과 job 완성되기 전, business logic을 실행할 수 있는 기회를 준다.

예를들면 환경설정 셋팅을 lob 전에 할 수 있고, job이 완성된 후 cleanup할 수 있다

 

 

package com.websystique.springbatch;

import java.util.List;

import org.joda.time.DateTime;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class ExamResultJobListener implements JobExecutionListener{
	 
    private DateTime startTime, stopTime;
 
    @Override
    public void beforeJob(JobExecution jobExecution) {
        startTime = new DateTime();
        System.out.println("ExamResult Job starts at :"+startTime);
    }
 
    @Override
    public void afterJob(JobExecution jobExecution) {
        stopTime = new DateTime();
        System.out.println("ExamResult Job stops at :"+stopTime);
        System.out.println("Total time take in millis :"+getTimeInMillis(startTime , stopTime));
 
        if(jobExecution.getStatus() == BatchStatus.COMPLETED){
            System.out.println("ExamResult job completed successfully");
            //Here you can perform some other business logic like cleanup
        }else if(jobExecution.getStatus() == BatchStatus.FAILED){
            System.out.println("ExamResult job failed with following exceptions ");
            List
 exceptionList = jobExecution.getAllFailureExceptions();
            for(Throwable th : exceptionList){
                System.err.println("exception :" +th.getLocalizedMessage());
            }
        }
    }
 
    private long getTimeInMillis(DateTime start, DateTime stop){
        return stop.getMillis() - start.getMillis();
    }
 
}

Step 7: Create Spring Context with job configuration

Src/main/resource/spring-batch-context.xml

 

 

 

 

 

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">
 
    <!-- JobRepository and JobLauncher are configuration/setup classes -->
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
 
    <bean id="jobLauncher"   class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
    </bean>
 
    <!-- ItemReader은  input file로부터 완성된 라인을 하나하나씩 읽는다. -->
    <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"  scope="step">
        <property name="resource" value="classpath:examResult.txt" />
 
        <property name="lineMapper">
 
            <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
 
                <property name="fieldSetMapper">
                    <!-- Mapper는  기록물 안에 있는 각각의 item(항목)을 POJO 속성에 map시킨다. -->
                    <bean class="com.websystique.springbatch.ExamResultFieldSetMapper" />
                </property>
 
                <property name="lineTokenizer">
                    <!-- A tokenizer class는 input record에 있는 항목을  특정 지은 문자("|")로 분리해서 사용하기 위한 클래스이다. -->
                    <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                         <property name="delimiter" value="|" />
                    </bean>
                </property>
 
            </bean>
 
        </property>
 
    </bean>
 
    <!-- XML ItemWriter which writes the data in XML format -->
    <bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
 
        <property name="resource" value="file:xml/examResult.xml" />
 
        <property name="rootTagName" value="UniversityExamResultList" />
 
        <property name="marshaller">
 
            <bean class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
                <property name="classesToBeBound">
                    <list>
                        <value>com.websystique.springbatch.model.ExamResult</value>
                    </list>
                </property>
            </bean>
 
        </property>
 
    </bean>
 
    <!-- Optional ItemProcessor to perform business logic/filtering on the input records -->
    <bean id="itemProcessor" class="com.websystique.springbatch.ExamResultItemProcessor" />
 
    <!-- Optional JobExecutionListener to perform business logic before and after the job -->
    <bean id="jobListener" class="com.websystique.springbatch.ExamResultJobListener" />
 
    <!-- Step will need a transaction manager -->
    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
 
    <!-- Actual Job -->
    <batch:job id="examResultJob">
        <batch:step id="step1">
            <batch:tasklet transaction-manager="transactionManager">
                <batch:chunk reader="flatFileItemReader" writer="xmlItemWriter"  processor="itemProcessor" commit-interval="10" />
            </batch:tasklet>
        </batch:step>
        <batch:listeners>
            <batch:listener ref="jobListener" />
        </batch:listeners>
    </batch:job>
 
</beans>

설정한 stemp1은 FlatFileItemReader로 기록물을 읽고, itemProcessor로 기록물을 처리하며, StarEventItemWriter을 이용해 기록물을 출력한다.

Commit-interval은 transaction이 committed되기 전/  출력이 발생하기 전 처리할 수 있는 item의 수를 구체화한다.

다수의 기록물을 하나의 transaction에 그룹 짓고,그것들을 chunk(덩어리)로 묶는다.

 

Step 8: Create Main application to finally run the job

 

package com.websystique.springbatch;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class App 
{
    public static void main( String[] args )
    {
        ApplicationContext context = new ClassPathXmlApplicationContext("spring-batch-context.xml");
        
        JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("examResultJob");
        
        try {
            JobExecution execution = jobLauncher.run(job, new JobParameters());
            System.out.println("Job Exit Status : "+ execution.getStatus());
 
        } catch (JobExecutionException e) {
            System.out.println("Job ExamResult failed");
            e.printStackTrace();
        }

    }
}

 

 

 

번역 :http://websystique.com/springbatch/spring-batch-read-a-csv-file-and-write-to-an-xml-file/