Spring Batch- Read an XML file and write to a CSV file

이번에는 StaxEventItemReader를 사용해서 XML file 읽고 FlatFileItemWriter를 사용해서 Flat CSV file을 쓰기위한 Spring Batch 사용법을 배울 것이다.

또한 JobExcutionListener과 itemProcessor 어법의 사용을 볼 수 있을 것이다.

시작해봅시다!

 

Following technologies being used:

• Spring Batch 3.0.1.RELEASE
• Spring core 4.0.6.RELEASE
• Spring oxm 4.0.6.RELEASE
• Joda Time 2.3
• JDK 1.6
• Eclipse JUNO Service Release 2

 

Step 1: Create project directory structure

 

(src/main/resources/examResult.xml) XML file을 읽고 (project/csv/ExamResult.txt) CSV 파일을 출력할 것이다

 

Step 2: Update pom.xml to include required dependencies

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.websystique.springbatch</groupId>
	<artifactId>SpringBatchXmlToCsv</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>SpringBatchXmlToCsv</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<springframework.version>4.0.6.RELEASE</springframework.version>
		<springbatch.version>3.0.1.RELEASE</springbatch.version>
		<joda-time.version>2.3</joda-time.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${springframework.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-oxm</artifactId>
			<version>${springframework.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${springbatch.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-infrastructure</artifactId>
			<version>${springbatch.version}</version>
		</dependency>
		<dependency>
			<groupId>joda-time</groupId>
			<artifactId>joda-time</artifactId>
			<version>${joda-time.version}</version>
		</dependency>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>3.8.1</version>
			<scope>test</scope>
		</dependency>
	</dependencies>
	   <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.2</version>
                    <configuration>
                        <source>1.6</source>
                        <target>1.6</target>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

Data-time 처리를 위해서 joda-time api를 사용할 것이다.

 

Step 3: Prepare the input XML file and corresponding domain object /mapped POJO

아래의 xml data를 flat file format으로 전환할 것이다.

 

<?xml version="1.0" encoding="UTF-8"?>
<UniversityExamResultList>
	<ExamResult>
		<dob>1985-02-01</dob>
		<percentage>76.0</percentage>
		<studentName>Brian Burlet</studentName>
	</ExamResult>
	<ExamResult>
		<dob>1970-02-01</dob>
		<percentage>61.0</percentage>
		<studentName>Renard konig</studentName>
	</ExamResult>
	<ExamResult>
		<dob>1993-02-01</dob>
		<percentage>92.0</percentage>
		<studentName>Rita Paul</studentName>
	</ExamResult>
	<ExamResult>
		<dob>1965-01-31</dob>
		<percentage>83.0</percentage>
		<studentName>Han Yenn</studentName>
	</ExamResult>
</UniversityExamResultList

위에의 파일 행 내용과 일치하는 필드롤 POJO에 mapping 시킬 것이다.

package com.websystique.springbatch.model;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;

import org.joda.time.LocalDate;

@XmlRootElement(name = "ExamResult")
public class ExamResult {
	private String studentName;
	private LocalDate dob;
	private double percentage;
	
	@XmlElement(name = "studentName")
	public String getStudentName() {
		return studentName;
	}
	public void setStudentName(String studentName) {
		this.studentName = studentName;
	}
	
	@XmlElement(name = "dob")
	@XmlJavaTypeAdapter(type = LocalDate.class, value = com.websystique.springbatch.LocalDateAdapter.class)
	public LocalDate getDob() {
		return dob;
	}
	public void setDob(LocalDate dob) {
		this.dob = dob;
	}
	public double getPercentage() {
		return percentage;
	}
	public void setPercentage(double percentage) {
		this.percentage = percentage;
	}
	
	 @Override
	    public String toString() {
	        return "ExamResult [studentName=" + studentName + ", dob=" + dob
	                + ", percentage=" + percentage + "]";
	    }

}

Class 속성을 XML tag에 map시키기 위해 JAXB annotations을 사용한 것이다.

Joda-time LocalDate API를 사용하기 떄문에 JAXB에게 어떻게 전환을 할 것인지 알려주어야 한다. 아래의 Adapter class가 그것을 위한 것이다

 

package com.websystique.springbatch;

import javax.xml.bind.annotation.adapters.XmlAdapter;

import org.joda.time.LocalDate;

public class LocalDateAdapter extends XmlAdapter
 {

	@Override
	public String marshal(LocalDate v) throws Exception {
		return v.toString();
	}

	@Override
	public LocalDate unmarshal(String v) throws Exception {
		return new LocalDate(v);
	}

}

 

Step 4: Create an ItemProcessor

ItemProcessor은 선택적이고, item(항목)을 읽은 후에 호출되어지고, item을 쓰기 전에 호출되어진다.

각각의 item에 business logic을 이행하기 위한 기회를 주기 위함이다.

예를들면, percentage가 75보다 작은 item을 제외할 수 있다(즉 percentage >= 75)

 

package com.websystique.springbatch;

import org.springframework.batch.item.ItemProcessor;

import com.websystique.springbatch.model.ExamResult;

public class ExamResultItemProcessor implements ItemProcessor
{

	@Override
	public ExamResult process(ExamResult result) throws Exception {
		System.out.println("Processing result :"+result);
		
		if(result.getPercentage() < 75){
			return null;
		}
		
		return result;
	}

}

 

Step 5: Add a Job listener(JobExecutionListener)

Job listener은 선택적이며,  job(작업)을 시작학 전에, 완료한 후에 business logic을 실행시킬 수 있다. 예를들자면, job 전에 환경설정 셋팅을 하거나 job 완료 후 cleanup할 수 있다.

 

 

package com.websystique.springbatch;

import java.util.List;

import org.joda.time.DateTime;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class ExamResultJobListener implements JobExecutionListener{
	 
    private DateTime startTime, stopTime;
 
    @Override
    public void beforeJob(JobExecution jobExecution) {
        startTime = new DateTime();
        System.out.println("ExamResult Job starts at :"+startTime);
    }
 
    @Override
    public void afterJob(JobExecution jobExecution) {
        stopTime = new DateTime();
        System.out.println("ExamResult Job stops at :"+stopTime);
        System.out.println("Total time take in millis :"+getTimeInMillis(startTime , stopTime));
 
        if(jobExecution.getStatus() == BatchStatus.COMPLETED){
            System.out.println("ExamResult job completed successfully");
            //Here you can perform some other business logic like cleanup
        }else if(jobExecution.getStatus() == BatchStatus.FAILED){
            System.out.println("ExamResult job failed with following exceptions ");
            List
 exceptionList = jobExecution.getAllFailureExceptions();
            for(Throwable th : exceptionList){
                System.err.println("exception :" +th.getLocalizedMessage());
            }
        }
    }
 
    private long getTimeInMillis(DateTime start, DateTime stop){
        return stop.getMillis() - start.getMillis();
    }
 
}

 

Step 6: Create Spring Context with job configuration

 

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">
 <!--JobRepository and JobLauncher 환경설정 및  classes 구성  -->
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
 
    <bean id="jobLauncher"   class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
    </bean>
 
    <!-- ItemWriter write a line into output flat file -->
    <bean id="flatFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
 
        <property name="resource" value="file:csv/examResult.txt" />
 
        <property name="lineAggregator">
 
            <!-- An Aggregator는 객채를 범위가 정해진 string의 list로 전환한다-->
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
 
                <property name="delimiter" value="|" />
 
                <property name="fieldExtractor">
 
                    <!-- Extractor는 리플렉션을 통해서 beans property 값을 리턴한다 -->
                    <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                        <property name="names" value="studentName, percentage, dob" />
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
 
    <!-- ItemReader는 XML file의 데이터를 읽는다 -->
    <bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">
 
        <property name="resource" value="classpath:examResult.xml" />
 
        <property name="fragmentRootElementName" value="ExamResult" />
 
        <property name="unmarshaller">
            <bean class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
                <property name="classesToBeBound">
                    <list>
                        <value>com.websystique.springbatch.model.ExamResult</value>
                    </list>
                </property>
            </bean>
        </property>
    </bean>
 
 
    <!-- Optional ItemProcessor to perform business logic/filtering on the input records -->
    <bean id="itemProcessor" class="com.websystique.springbatch.ExamResultItemProcessor" />
 
    <!-- Optional JobExecutionListener to perform business logic before and after the job -->
    <bean id="jobListener" class="com.websystique.springbatch.ExamResultJobListener" />
 
    <!-- Step will need a transaction manager -->
    <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
 
    <!-- Actual Job -->
    <batch:job id="examResultJob">
        <batch:step id="step1">
            <batch:tasklet transaction-manager="transactionManager">
                <batch:chunk reader="xmlItemReader" writer="flatFileItemWriter"
                    processor="itemProcessor" commit-interval="10" />
            </batch:tasklet>
        </batch:step>
        <batch:listeners>
            <batch:listener ref="jobListener" />
        </batch:listeners>
    </batch:job>
</beans>

 

설정한 stemp1은 FlatFileItemReader로 기록물을 읽고, itemProcessor로 기록물을 처리하며, StarEventItemWriter을 이용해 기록물을 출력한다.

Commit-interval은 transaction이 committed되기 전/  출력이 발생하기 전 처리할 수 있는 item의 수를 구체화한다.

다수의 기록물을 하나의 transaction에 그룹 짓고,그것들을 chunk(덩어리)로 묶는다.

 

Step 7: Create Main application to finally run the job

package com.websystique.springbatch;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;


public class App 
{
	@SuppressWarnings("resource")
    public static void main(String areg[]){
 
        ApplicationContext context = new ClassPathXmlApplicationContext("spring-batch-context.xml");
 
        JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("examResultJob");
 
        try {
            JobExecution execution = jobLauncher.run(job, new JobParameters());
            System.out.println("Job Exit Status : "+ execution.getStatus());
 
        } catch (JobExecutionException e) {
            System.out.println("Job ExamResult failed");
            e.printStackTrace();
        }
    }

}

 

 

번역:http://websystique.com/springbatch/spring-batch-read-an-xml-file-and-write-to-a-csv-file/