■ INTRODUCTION xxii THE EXPERT’S VOICE ® IN SPRING Dr. Mark Lui, Mario Gray, Andy H. Chan, and Josh Long Build enterprise integration solutions using Spring Integration Pro Spring Integration www.it-ebooks.info For your convenience Apress has placed some of the front matter material after the index. Please use the Bookmarks and Contents at a Glance links to access them. www.it-ebooks.info iv Contents at a Glance Contents v About the Authors xvi About the Technical Reviewers xvii Acknowledgments xviii Introduction xix ■Chapter 1: Enterprise Application Integration Fundamentals 1 ■Chapter 2: Exploring the Alternatives 15 ■Chapter 3: Introduction to Core Spring Framework 61 ■Chapter 4: Introduction to Enterprise Spring 103 ■Chapter 5: Introduction to Spring Integration 149 ■Chapter 6: Channels 175 ■Chapter 7: Transformations and Enrichment 217 ■Chapter 8: Message Flow: Routing and Filtering 245 ■Chapter 9: Endpoints and Adapters 291 ■Chapter 10: Monitoring and Management 329 ■Chapter 11: Talking to the Metal 361 ■Chapter 12: Enterprise Messaging with JMS and AMQP 397 ■Chapter 13: Social Messaging 451 ■Chapter 14: Web Services 477 ■Chapter 15: Extending Spring Integration 499 ■Chapter 16: Scaling Your Spring Integration Application 529 ■Chapter 17: Spring Integration and Spring Batch 561 ■Chapter 18: Spring Integration and Your Web Application 591 Index 615 www.it-ebooks.info xix Introduction The majority of the open source Java frameworks have focused on supporting database-backed web sites. Developers have leveraged these frameworks to create highly scalable and performant vertical applications using projects such as Spring and Hibernate. Recently, a number of frameworks have been developed with the purpose of solving the horizontal problem of integrating data and services between disparate applications across the enterprise. Spring Integration is one of these solutions. Enterprise integration is an architectural approach for integrating disparate services and data in software. Enterprise integration seeks to simplify and automate business processes without requiring comprehensive changes to the existing applications and data structures. Spring Integration is an extension of Spring’s Plain Old Java Object (POJO) programming model to support the standard integration patterns while building on the Spring Framework’s existing support for integration with systems and users. The Spring Framework is the most widely used framework in organizations today. Spring Integration works in terms of the fundamental idioms of integration, including messages, channels, and endpoints. It enables messaging within Spring-based applications and integrates with external systems via Spring Integration’s adapter framework. The adapter framework provides a higher level of abstraction over Spring’s existing support for remote method invocation, messaging, scheduling, and much more. Developers already familiar with the Spring Framework will find Spring Integration easy to pick up, since it uses the same development model and idioms. This book will cover the vast world of enterprise application integration, as well as the application of the Spring Integration framework toward solving integration problems. The book can be summed up as the following: • An introduction to the concepts of enterprise application integration. • A reference on building event-driven applications using Spring Integration. • A guide to solving common integration problems using Spring Integration. Who This Book Is For This book is for any developer looking for a more natural way to build event-driven applications using familiar Spring idioms and techniques. The book is also for architects seeking to better their applications and increase productivity in their developers. You should have a basic understanding of the Java language. A familiarity with the Spring Framework and messaging are useful, but not required, as we provide an introduction to those concepts. www.it-ebooks.info ■ INTRODUCTION xx How This Book Is Structured To give you a quick idea of what this book covers, here’s a brief chapter-by-chapter overview: • Chapter 1 provides an introduction to enterprise application integration—an architectural approach for integrating disparate services and data in software. • Chapter 2 takes a look as some of the alternative open source technologies for enterprise integration. • Chapter 3 introduces the Spring Framework and provides a glance at some of the major modules leveraged by Spring Integration. • Chapter 4 introduces several of the enterprise APIs supported by the core Spring Framework, including those for JDBC, object-relationship management, transactions, and remoting. • Chapter 5 introduces the basic Spring Integration components and how they extend the Spring Framework into the world of messaging and event-driven architectures. • Chapter 6 introduces the concept of a message channel and how the Spring Integration framework simplifies the development of message channels in an integration implementation. • Chapter 7 covers transformations, which allow the message payload to be modified to the format and structure required by the downstream endpoint. It also covers enrichment, which allows for augmenting and modifying the message header values as required for supporting downstream message handling and endpoint requirements. • Chapter 8 covers the different components available for controlling message flow in Spring Integration, and how to use a workflow engine with Spring Integration. • Chapter 9 focuses on endpoints that connect to the message channels, application code, external applications, and services. • Chapter 10 introduces the support for management and monitoring provided by Spring Integration and other available open source systems. • Chapter 11 discusses the Spring Integration channel adapters used to communicate with files, sockets, streams, file servers, and databases. • Chapter 12 focuses on enterprise messaging using transports such as Java Message Service (JMS) and the Advanced Message Queuing Protocol (AMQP). • Chapter 13 discusses the Spring Integration adapters that support integrating with e-mail, XMPP (Jabber, GTalk, Facebook Chat, etc.), news feeds (RSS, ATOM, etc.), and Twitter. • Chapter 14 covers how Spring Integration provides both client and server support for web services and how to integrate web services with the Spring Integration messaging framework. • Chapter 15 explores the support offered in the core Spring Integration framework to help you build your own adapters. • Chapter 16 focuses on how to increase performance by scaling out the hardware and how Spring Integration applications can take advantage of concurrency. www.it-ebooks.info ■ INTRODUCTION xxi • Chapter 17 reviews the Spring Batch project and how it can be used with Spring Integration. • Chapter 18 shows how Spring Integration can be used to create a basic web interface with the HTTP inbound and outbound gateways. It also discusses the Spring Integration support for the server pushing data to the client browser. Conventions Sometimes when we want you to pay particular attention to a part within a code example, we will make the font bold. Please note that the bold does not necessarily reflect a code change from the last version. In cases when a code line is too long to fit the page’s width, we break it with a code continuation character, which looks like this: . Please note that when you type out the code, you have to concatenate the line by yourself without any spaces. Prerequisites Because the Java programming language is platform independent, you are free to choose any supported operating system. However, some of the examples in this book use platform-specific paths. Translate them as necessary to your operating system’s format before typing out the examples. To make the most of this book, install JDK version 1.5 or higher. You should have a Java IDE installed to make development easier. For this book, the sample code is Maven based. If you’re running Eclipse and you install the m2eclipse plug-in, you can open the same code in Eclipse, and the class path and dependencies will be filled in the by the Maven metadata. If you’re using Eclipse, you might prefer SpringSource’s free SpringSource Tool Suite (STS) (www.springsource.com/developer/sts), which comes preloaded with all the plug-ins you will need to be as efficient as possible with the Spring Framework. If you use NetBeans or IntelliJ IDEA, there are no special configuration requirements: they already support Maven out of the box and also provide some support for Spring. This book uses Maven (2.2.1 or higher). The recommended approach is to simply use a tool like Maven, Ant, Ivy, or Gradle to handle dependency management. The Maven dependency coordinates given in this book can also be used with Ivy, Gradle, and others. Downloading the Code The source code for this book is available from the Apress web site (www.apress.com), in the Source Code section. The source code is organized by chapters, each of which includes one or more independent examples. Note that there are four Spring Integration sandbox projects used in this book. Sandbox projects tend to be a moving target and we will do our best to keep the information up to date at the Apress web site for any changes required to build these projects. More information about build the sandbox project may be found in the readme.txt file within the source code. Contacting the Authors We always welcome your questions and feedback regarding the content of this book. You can reach Mark Lui via his blog at http://drmarklui.wordpress.com, or via e-mail at dr.mark.lui@gmail.com. Mario Gray can be contacted through his blog at http://www.sudoinit5.com, or by e-mail at mario@sudioinit5.com. Andy Chan can be reached via Twitter (@iceycake) or his blog site, http://www.iceycake.com. Josh Long can be reached at his blog at http://www.joshlong.com, by e-mail at josh@joshlong.com, or on Twitter (@starbuxman). www.it-ebooks.info C H A P T E R 1 ■ ■ ■ 1 Enterprise Application Integration Fundamentals Enterprise Application Integration (EAI) grew out of the incompatibility between the many different ERP applications prevalent during the 1990s and the many in-house applications that needed to use them. ERP applications like SAP, PeopleSoft, and JD Edwards, and customer relationship management systems (CRMs) like Clarify and Siebel quickly became enterprise data silos, and it became increasingly important to reuse the data and functionality in those systems. EAI is an architectural approach for integrating disparate services and data in software. EAI seeks to simplify and automate business processes without requiring comprehensive changes to the existing applications and data structures. As an organization grows in size, it creates different departments with particular areas of focus, interests, and expertise. Partitioning is required to keep team sizes down to a manageable level and to foster hires of the best people for a particular set of responsibilities while providing enough autonomy to get the work done. While the instinct to partition is natural, all departments must also work together, sharing business processes and data in service of an overall goal and vision. Business processes develop over time, organically; some focus on the entire enterprise as a whole and some are unique to a particular area. Software applications are developed or purchased to support business processes. Organizations can end up with a wide range of sometimes overlapping, conflicting, or incompatible applications and systems. These applications may be based on different operating systems, may use different supporting databases, or be written in different computer languages. It can be very difficult to bridge these disparate applications and services, owing in part to technical incompatibilities and to the prohibitive costs of cross-training personnel in the various systems. Integration of Data and Services Between Disparate Systems The main driver for EAI is the desire to share data and business processes across existing applications within an enterprise. In a typical scenario, a company purchases a new CRM system (see Figure 1–1). This system is a major upgrade from the old homegrown mainframe system that has served the company well for many years. It is a common requirement to simultaneously employ the new system while still keeping the old system in service, usually because certain of the old system’s functions aren’t yet available in the new one. In such a scenario, new customer information is entered into the CRM but the legacy system must be synchronized to reflect the data to support required enterprise business processes. In addition, the CRM might also need to invoke some of the legacy system’s functions. www.it-ebooks.info CHAPTER 1 ■ ENTERPRISE APPLICATION INTEGRATION FUNDAMENTALS 2 Figure 1–1. Integration with Modern CRM and Legacy System Organizations often want to use the best of breed software solution. Although it is possible to purchase software applications from a single vendor to support a large enterprise’s needs, organizations usually like to use what is considered the best software for each business function. This happens for several reasons: specialization sometimes yields better results, and it’s always practical to avoid completely investing in one vendor. Support could be an issue if the vendor goes out of business. In addition, there is the possibility that only a custom application will fulfill the business needs. Even with modern systems there may not be a standard means of communication that would work with each vendor’s software. Business applications can run on different operation systems such as Linux, Mac, Windows, Solaris, HP-UX, and IBM-AIX. These applications may be based on different databases, such as Oracle, DB2, SQL server, Sybase, RDB, and Informix. Applications are written in different languages such as Java, C, C++, .NET, Cobol, and ABAP. In addition the legacy mainframe systems (e.g., IBM and DEC) should not be forgotten. Integration Between Information Silos An information silo is defined as a management system that is incapable of reciprocal operation with other, related management systems. The focus of even internally built applications is inward, and the communication emphasis is vertical. Even with the current push toward open standards and the desire to exploit the power of the Internet, information silos are quite prevalent in most organizations. They are usually caused by the lack of communication and common goals between departments in an organization. Surveying the majority of the current open source framework, the focus is toward vertical database-backed web applications. There are fewer options for horizontal communication between these applications. This may be another driver, or just be symptomatic of the growth of information silos. Information silos limit the ability to achieve business process interoperability and prevent an organization from leveraging all the departments to work toward a common goal. In addition, it prevents the applications from using the full power of the internet. Integration between information silos is another problem that EAI attempts to resolve. Integration Between Companies With the power and promise of the Internet, an opportunity arose to improve communication between different companies. After the initial focus on business-to-consumer (B2C) applications, a movement took place to create business-to-business (B2B) systems. Electronic information exchange between different organizations has the potential to reduce cost, increase efficiency, and eliminate human error. The electronic data interchange (EDI) standard was created to support electronic commerce transactions between two computer systems or trading partners. EDI uses a series of electronic messages sent from one computer system to another without human intervention. The message must abide by strictly defined contracts. EDI’s original intent was to create a common message standard for B2B communication. EDI is still heavily used, especially within the financial industry. Although promising, EDI posed a number of issues stemming from a lack of a general purpose message format. EDI lacks obvious looping declarations. The separation of structure from the data increases the difficulty in extracting the proper information. EDI has no standard parsing API, and usually requires proprietary software. www.it-ebooks.info CHAPTER 1 ■ ENTERPRISE APPLICATION INTEGRATION FUNDAMENTALS 3 Other industries also feature standards to facilitate partner communication. In the healthcare industry in the United States, for example, HL7 describes the secure exchange of patient health information and treatment. In contrast, XML was created as a general-purpose data format that can be easily transmitted over the foundations of the Internet, including HTTP. XML has had a broad adoption and has become the de-facto standard for message exchange. Web services are a response to the need to expose business processes using HTTP communication. The growth of web services speaks to how important integration is. Integration with Users Today’s applications aren’t mainframe applications with amber-screen clients. They are web applications. Where “web application” might’ve described an application accessible from a web browser five years ago, today it describes an application that stores your information in one place and exposes its feature sets in many ways. Today’s users increasingly rely on their tablet computers, their mobile devices, and their everyday tools—chat, e-mail, news feeds, and social networks, like Facebook or Twitter—to interact with these applications and with each other. Today’s developer can no longer afford to expect the user to be at a desk, logged into a web page. She must go to where the user is and make it as easy as possible for those users to use the application. This is integration, and it’s a key part of the most radical changes in application developments in the last five years. Challenges EAI implementations do not have the best reputation for success. There have been many reports that indicate the majority of EAI projects fail. And the failures are not usually due to software or technical issues, but to management challenges. Before getting into the management issues with EAI, a look the technical issues is appropriate. Technology EAI was originally dominated by the commercial vendors offering a number of proprietary solutions. Implementing an EAI solution required a strong knowledge of the vendor’s software and tools. A number of the commercial products and approaches will be discussed later in this chapter. Until recently, viable open source frameworks did not exist. Although open source integration was possible, the results lacked the required management and monitoring capabilities for a production environment. In addition, these solutions required a great deal of custom coding of business logic and adapters to many of the existing enterprise applications. Spring Integration is one open source solution that has come of age and is ready to solve your integration requirements. Alternative open source solutions are discussed in Chapter 2. Integration usually requires interfacing with a number of different technologies and business domains. SAP may be used for accounting, Siebel for customer relations, and PeopleSoft for human resources. All these applications have different technologies, configurations, and external communication protocols. It is usually very difficult to integrate with an application or system without knowledge of the underlying endpoint system. Adapter technology offsets some of these difficulties, but often there needs to be some configuration and software modifications to the applications to integrate. Different systems expose different integration options: SAP has business application programming interface (BAPI), Siebel has business components, and PeopleSoft has business objects. These systems are formidable, but usually require a system integrator to have some basic understanding of the applications with which to be connected, and the business domain with which they are part. Depending on the number of integration points, this can be quite a challenge. Many times the target application cannot be changed or altered to enable the integration. The application may be a commercial product in which any changes would void the warranty and support www.it-ebooks.info CHAPTER 1 ■ ENTERPRISE APPLICATION INTEGRATION FUNDAMENTALS 4 process. Or the application could be legacy with which any change is difficult. Despite the widespread need for integration standards, only a few have emerged. XML and web services have received a great deal of hype, but these technologies are still marked with a large amount of fragmentation. Even with numerous standards organizations in the world today the specifications are becoming more and more inconsistent. Look at the number of Web Services (WS-*) standards and the number of corresponding Java specification requests (JSRs). The standards committees are typically dominated by the vendors with their own agenda. On a positive note, however, the open-source community more and more drives standards adoption. Today’s applications are often a myriad of many moving parts, separated by unknown networks and protocols. It is foolish to assume that both client and server will always be available. By using messaging to introduce asynchronous communication between two systems, you can not only decouple the producer from the uptime—after all the messages are buffered until both sides have capacity to handle it—you can also speed up the performance of individual components in the system since they no longer have to wait for each other before proceeding. Similarly, independent parts of a system may be scaled out to achieve capacity. People As with all business problems, solving the technical issues is the easy part. Dealing with the people issues is what’s hard. It’s no different with EAI. Implementing it may even be more difficult than implementing a vertical application, since it runs across the entire enterprise. EAI implementations often cross corporate boundaries, engage partners, and touch customers. This is where the real fun begins when internal corporate politics start entering into the mix causing simple questions to take months to resolve. One multinational company had a different set of software standards and practices dependent on which side of the Atlantic the division was located. Integrations typically touch many parts of an organization with a different set of technologies as well as processes, management styles, and politics. The motivation for the integration implementation may be different for the various areas. Often one area may not want to share data with the other. A successful integration requires that there can be an agreement on what data will be shared, in what format, and how to map the different representations and interpretations between the areas. This in not always easy or achievable, and many times compromises must be reached. The timing for implementing integration may be determined by different factors across an organization. One area might want the data made available as soon as possible where the other may be working on another project and have no motivation for the integration effort. This needs to be negotiated, because integration with an application usually requires access and support by the business owners. Security is always an issue, because data may be proprietary because of privacy and/or business value. Integration requires access to this data and obtaining the appropriate authorization is not always easy. However, this access is vital to the success of the implementation where security requirement must be met. A successful integration often requires both communication between computers systems as well as business units and information technology departments. Because of there wide scope, integration efforts have far reaching implications on the business. Failed integration processes can cost a business millions of dollars in lost orders, misrouted payments, and disgruntled customers. In the end however, working well with everyone from the business owners to the computer support staff is essential and often more important than the technical issues involving an integration. Integration frameworks such Spring Integration mitigate the technical barriers, but you must be sure to address business motivations as well. www.it-ebooks.info [...]... using an integration framework (see Figure 1–12) Figure 1–12 Workflow Pattern Spring Integration Framework The Spring Integration Framework is a response to the need for an open-source, straightforward integration framework that leverages the widely adopted Spring Framework Spring Integration provides an extension of Spring s plain old Java object (POJO) programming model to support the standard integration. .. approach works Domination by Proprietary Solutions EAI solutions have traditionally been dominated by costly proprietary solutions that required domain experts to implement Until recently, these have been the best approach to solving integration problems Some of the more popular integration products are discussed in the following sections webMethods (Active Software) webMethods is an integration product... com.apress.prospringintegration.jms http-consumer-su 1.0-SNAPSHOT com.apress.prospringintegration.jms jms-provider-su 1.0-SNAPSHOT com.apress.prospringintegration.jms jms-consumer-su... approach Before diving headfirst into Spring Integration, we’ll explore several of the available and popular open source integration frameworks The two most popular projects are Mule ESB (www.mulesoft.org) and Apache ServiceMix (http://servicemix.apache.org) Each of these projects takes a different approach This chapter will cover four different integration approaches: • Mule provides a lightweight container... building on the Spring Framework’s existing support for enterprise integration The Spring Framework is the most widely adapted framework used in organizations today Spring Integration works in terms of the fundamental idioms of integration, including messages, channels, and endpoints It enables messaging within Spring- based applications and integrates with external systems via Spring Integration s adapter... components of webMethods are the Integration Server and message broker The Integration Server is the predecessor to the modern application server Essentially an HTTP server on steroids, the Integration Server was developed to support B2B integration With support for EDI and custom XML messages, and its own process flow language, Integration Server provides a platform to integration between companies over... xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans /spring- beans-2.0.xsd http://activemq.apache.org/camel/schema /spring http://activemq.apache.org/camel/schema /spring/ camel -spring. xsd"> ... webMethods added an enterprise integration platform to its offerings Active Software products included an enterprise grade message broker and a suite of adapters supports integration with all the major ERP and legacy systems Additional acquisitions added monitoring and business process support, and created a complete integration platform Commercial products such as webMethods provide a suite of visual tools... accepted open source integration frameworks, Mule ESB was the result of Ross Mason’s system integration consulting work looking for a method to increase productivity and maintainability of custom integration engagements Philosophy and Approach Mule’s approach was to create a lightweight container that can run standalone, without requiring the support of a J2EE application server where the integration component... command: mvn mule-project-archetype:create -DartifactId=mule-example -DmuleVersion=3.0.0 Maven will prompt with a number of questions Enter Mule Example for the project description, enter com/apress/prospringintegration/mule for the Java package path, enter http,jms,vm for the Mule transports, and accept the defaults for the rest of the questions The archetype will create a basic Mule project with the . VOICE ® IN SPRING Dr. Mark Lui, Mario Gray, Andy H. Chan, and Josh Long Build enterprise integration solutions using Spring Integration Pro Spring Integration www.it-ebooks.info For. 15: Extending Spring Integration 499 ■Chapter 16: Scaling Your Spring Integration Application 529 ■Chapter 17: Spring Integration and Spring Batch 561 ■Chapter