Comments on: The EJB Specification, Concurrency, and Batch Processing http://javablog.franksalinas.net/2009/03/01/the-ejb-specification-concurrency-and-batch-processing/ Java Enterprise Development & Technology Wed, 10 Mar 2010 23:24:13 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Snehal Antani http://javablog.franksalinas.net/2009/03/01/the-ejb-specification-concurrency-and-batch-processing/comment-page-1/#comment-314 Snehal Antani Sun, 15 Nov 2009 18:39:45 +0000 http://javablog.franksalinas.net/?p=211#comment-314 An interesting design point to consider is that the degree of parallelism should be a point-in-time decision, influenced by the available capacity of the system, service-level agreements (SLA's) of concurrent workloads (OLTP, Real-Time, etc), approaching deadlines for other batch jobs in execution, and so on. It's easy for a developer to arbitrarily spin off 10 commonj threads for parallel execution, but 10 threads may be too few or too many given the other workloads in the system. For designing parallel processing for batch, design the application to not care about the degree of parallelization. Shift the burden of assessing the degree of parallelization to the infrastructure, where some external component applies a partitioning algorithm to the job, dispatches the many instances of the job across the collection of application server threads, where each job instance operates on its own segment of the data. This is how we designed it in WebSphere Compute Grid, IBM's batch processing technology. The following article and presentation discuss batch application design patterns. They are generally technology agnostic, though the examples are provided in the context of WebSphere Compute Grid. - Batch application design: - http://sites.google.com/site/snehalantani/designingBatchApps.zip - http://sites.google.com/site/snehalantani/DesigningBatchApplications.pdf Batch processing infrastructure overview: - http://sites.google.com/site/snehalantani/WebSphereDataIntensiveApps.pdf - http://www-128.ibm.com/developerworks/websphere/techjournal/0804_antani/0804_antani.html Customer experience with modern batch processing: - http://www-01.ibm.com/software/tivoli/features/ccr2/ccr2-2008-12/swissre-websphere-compute-grid-zos.html Latest Compute Grid overview presentation: - http://sites.google.com/site/snehalantani/latestpresentationmaterial An interesting design point to consider is that the degree of parallelism should be a point-in-time decision, influenced by the available capacity of the system, service-level agreements (SLA’s) of concurrent workloads (OLTP, Real-Time, etc), approaching deadlines for other batch jobs in execution, and so on. It’s easy for a developer to arbitrarily spin off 10 commonj threads for parallel execution, but 10 threads may be too few or too many given the other workloads in the system. For designing parallel processing for batch, design the application to not care about the degree of parallelization. Shift the burden of assessing the degree of parallelization to the infrastructure, where some external component applies a partitioning algorithm to the job, dispatches the many instances of the job across the collection of application server threads, where each job instance operates on its own segment of the data. This is how we designed it in WebSphere Compute Grid, IBM’s batch processing technology.

The following article and presentation discuss batch application design patterns. They are generally technology agnostic, though the examples are provided in the context of WebSphere Compute Grid.

- Batch application design:
- http://sites.google.com/site/snehalantani/designingBatchApps.zip
- http://sites.google.com/site/snehalantani/DesigningBatchApplications.pdf

Batch processing infrastructure overview:
- http://sites.google.com/site/snehalantani/WebSphereDataIntensiveApps.pdf
- http://www-128.ibm.com/developerworks/websphere/techjournal/0804_antani/0804_antani.html

Customer experience with modern batch processing:
- http://www-01.ibm.com/software/tivoli/features/ccr2/ccr2-2008-12/swissre-websphere-compute-grid-zos.html

Latest Compute Grid overview presentation:
- http://sites.google.com/site/snehalantani/latestpresentationmaterial

]]>