Showing posts with label ultraesb. Show all posts
Showing posts with label ultraesb. Show all posts

Tuesday, August 27, 2013

AdroitLogic announces v2.0.0 of the UltraESB

The version 2.0.0 of the UltraESB is a major release, influenced by requirements of one of the top Fortune #10 companies


Singapore - August 27, 2013 - AdroitLogic Private Ltd. announced today the release v2.0.0 of its free and open source Enterprise Service Bus, UltraESB. The UltraESB is released under the OSI approved GNU Affero General Public License; as well as a zero-dollar commercial license. First released in January 2010, the UltraESB was the first ESB to utilize zero-copy proxying, coupled with non-blocking IO to provide extreme levels of performance.

This is a major release which includes many enhancements to support large scale deployments, and has been influenced to meet requirements of a top Fortune #10 company. Some of the key features include a new built-in metrics management and alerting functionality, coupled with the concept of isolated deployment units and the ability to re-load changes atomically, including changes to JAR files associated with a deployment unit, without a re-start of the ESB while the ESB continues to process messages. A new interceptor framework allows user code to intercept messages for custom processing, using annotations. This is coupled with a new programmatic server management API to query and programmatically control the server configurations at runtime.

A concept of environments has been introduced to tweak the system with ease, using the factory defined environments already tweaked for a set of named profiles. These built-in profiles include dev, test, stage, prod etc.. making it easy to perform life cycle management of the ESB. The modules of the ESB has been re-organized, keeping only the most commonly used features in the core while optional functions and features have been moved to the optional module.

The user API has been enhanced with improved mediation support interfaces and a custom feature initializer framework, allowing a user to override default implementations. The enhancements to the API includes the introduction of the support interfaces for transports, decoupling the user from having to be aware of transport related properties, while backwards compatibility is maintained.

The UTerm (Command Line Interface) commands have been made consistent, making all commands subject first. For example, the earlier command to "list proxy services" (lps) now changes to "proxy service list" (psl) from 2.0.0 onwards. Some of the other changes include an improved AS2 persistence layer re-designed to support multi-tenancy, along with the upgrade of the Spring framework dependency to version 3.1.1.

The v2.0.0 release is approximately ~41MB in size and includes over 70 samples, test utilities and documentation etc to get started quickly. The release can be downloaded from http://adroitlogic.org

 

Monday, August 8, 2011

ESB Performance Testing.. Getting ready for the next round

Its been sometime since we've compared the performance of free and open source ESBs.. and now its time for the next round..

Amazon EC2 AMI to be published!
At the end of this round, we will publish an Amazon EC2 machine image (AMI) with the selected ESBs so that any end-user can simply fire up an instance of this image, and re-run the performance suite to compare the ESBs of his choice!

Read all about it here on ESB Magic and at http://esbperformance.org  

Sunday, June 5, 2011

Monitoring UltraESB with Zabbix

UltraESB is an open source project geared towards developing the worlds best ESB! It is already the best in performance, ease of use, and many features, but we are keen to further improve its core features for mediation, management and monitoring!

While working on the management/monitoring aspects of the UltraESB we have been introduced to Zabbix by one of our users. We already had a good web based management console called the UConsole, and a JSON REST services API for managing and monitoring, with a command line client to be released in the near future. Even though there will be many possibilities for management, each of these uses the same JMX infrastructure underneath. After all, JMX is supposed to be the management framework for all Java based applications.

Now, coming back to the topic, Zabbix monitoring has also been integrated with the UltraESB via the JMX API, facilitated by a native Zabbix agent that ships with the UltraESB v1.4.x onwards. The latest article is on the basics of monitoring the UltraESB using Zabbix.


Sometime back we mentioned about a set of great open source projects that the UltraESB utilizes, and we will soon be listing the free and open source Zabbix monitoring solution there! This again emphasizes the goal of Adroitlogic which is to build the best ESB out there, and not to re-implement all the existing wheels in the process! We will always select the best open source projects/tools out there which is not our core competency, and say publicly that they are what we use. Of course we do it without re-branding them as our own components the way some vendors sell others' open source projects as new products these days :)

Thursday, May 5, 2011

UltraESB Code Quality Metrics - updated for v1.4.0

Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skilful execution; it represents the wise choice of many alternatives.” - William A. Foster

UltraESB v1.4.0 - maintains 43.3% code coverage with samples that double in as unit tests; and 15.5% code comments resulting in a 46% documented API

Read about this ~35MB Open Source Enterprise Service Bus at it at ESB Magic!

Tuesday, May 3, 2011

Introducing the UConsole

UConsole is the UltraESB web management console. It is used to monitor the server state, artifacts and lets you mange them as well. It uses a set of bleeding edge technologies to bring the web management of the UltraESB into the table with HTML5 and CSS3 constructs to implement a modern web console with front end processing to render the pages.

UConsole Architecture

The architecture of the UConsole begins from the JMX in the UltraESB back-end where the actual management and monitoring aspects executed via the newly re-factored MXBeans. We have converted from MBeans to MXBeans because of the easiness of management with the MXBeans and the ability to serve complex data as CompositeData objects which can be deconstructed at the management console level easily to provide the management services.


These management services are completely restful and uses the JSON as the media type for the communication. In implementing that we have used Apache Wink deployed on Jetty. So when ever you login to the UltraESB web management console the browser front-end talks to the Wink application serving a set of JSON admin services running on Jetty. These admin services then talks to the UltraESB server via JMX and serves the request from the browser with the content loaded via JMX from the actual UltraESB server.

Now if you look at the browser front-end it is designed to be completely asynchronous and uses AJAX to talk to the Wink JSON services. We mainly use JavaScript with heavy use of jQuery as the key JavaScript framework. However the key piece of code that does the rendering from JSON data received from the Wink application into HTML is PURE and it is really capable of rendering JSON into HTML. At the front end we have been heavily using DataTables to render all the tabular data while using many other jquery.ui extensions to improve the console usability.

The authentication and authorization has been handled by the Apache Shiro framework, which registers as a servlet filer to the Wink RestServlet. That was the uconsole architecture in brief, and the above diagram shows the structure of the console.

UConsole in-action

The management console with the above simple nice architecture gives you a very valuable set of management capabilities for the UltraESB. Assuming that you are running a local UltraESB instance with the console on the same physical machine (which can be started with the uconsole.bat or uconsole.sh scripts available in the bin directory of the UltraESB), it is capable of auto-detecting that node. To manage the locally running node you need to login to the console.
Use the default username and password pair to login to the console which are "admin" and "admin" respectively. If you are managing a remote UltraESB server you need to specify the JMX URL of that UltraESB server on the appropriate input box with the user name and password.

Once you login, you will be able to see a dashboard with different graphs showing the state of the UltraESB server to which you are connected to. There will be 2 more tabs giving you the information on the system and a good set of references for getting started with the UltraESB. One important feature of the UConsole is, it's context sensitive help. If you click on the help icon right next to the heading of any page you will be provided with a nice context sensitive help as shown below;


If we look at the capabilities of this handy console, we better go through the navigation menu of the console to get an understanding of the each and every page it presents.

The first item is the dashboard that we have already talked about and the rest of it can be found in a nice colapsible menu. If you expand that menu, you will be able to see 2 major categories except for the Log viewer, which presents the tail of the log rendered into the console. The 2 major divisions of the UConsole is Node Management and Cluster Management while node management enabled you to manage artifacts on the logged in UltraESB server node, the cluster management section helps you to manage/monitor the complete cluster of the UltraESB from a single console. This comes in very handy when you have a cluster of UltraESBs and you want the central point of management for all the nodes in the cluster.

Under node management, you will get into a tabular list view of all the listed artifact types like, proxy services, sequences, endpoints and so forth. Do not forget to check on the context sensitive help on those pages to get some help as to what each and every page is intended to and what operations you can invoke from those pages. Apart from that, on the top right corner of the detailed artifact pages you will be able to see a control panel and you can use that control panel to invoke the operations for that artifact.

Moving fast to the cluster management, you will only be able to see the submenu section of the cluster management if you have clustering enabled, you may refer to my previous blog or the Clustered deployment of UltraESB article on how to enable clustering and run an UltraESB cluster. This section gives you the complete control over the cluster, where you can navigate through the nodes, to sessions in the cluster and gives a command history view. The most important bit is the control panel and it allows you to issue the cluster controlling commands from the UConsole and monitor those command states, and who has executed that command and so forth.

That is about the usage. Now if you look at the console header banner and the top right corner you will be able to see the Sign Out an another important feature named Switch Server switch server can be used to switch to another UltraESB node from the one you have been managing. While it can be used to switch to any UltraESB server if you are logged into a node in a given cluster, you will get the liberty of selecting a node in the cluster without typing it in.

So this turned out to be a bit long blog, but still I have many more to share with you all. Anyway you need to have a look at this awesome piece of software and get a feel of it. Download it now to get started with it.

Monday, May 2, 2011

AdroitLogic announces v1.4.0 of the UltraESB

The latest version brings support for clustering using ZooKeeper, and introduces a web based administration console, and support for Caching, XACML and JSON among many other features.

Singapore - May 2, 2011 - AdroitLogic Private Ltd. announced today the release v1.4.0 of its free and open source Enterprise Service Bus, the UltraESB. The UltraESB is released under the OSI approved GNU Affero General Public License; as well as a zero-dollar non-GPL commercial license which allows unlimited and perpetual use free of charge. First released in January 2010, the UltraESB was the first ESB to utilize memory mapped files and zero-copy, coupled with non-blocking IO to provide extreme levels of performance.

Version v1.4.0 released today offers support for clustering and management of cluster nodes through the new Web based administration console based on JMX. Clustering support is built over Apache ZooKeeper, which is used to manage extremely large clusters of Hadoop nodes. Advanced features such as automatic restart of a complete cluster of nodes - with round-robin processing, or management of a service, or endpoint across a cluster have been made trivial operations. The new version also supports caching, and the current release ships support for ehCache. The new release also ships support for high performance XML processing, utilizing FastXML. This allows end users to integrate third party libraries such as VTD XML with the UltraESB for extremely fast XPath processing, without parsing XML payloads. XACML is supported for fine grained authorization using the PicketBox  XACML library, and automatic conversion to/from JSON to XML makes processing JSON payloads even simpler.

In addition to supporting many transports and message formats, the UltraESB also offers support to easily develop JSON based data services with just a single line of configuration, and provides full support for the HTTP transport, including true REST support, basic, digest, NTLM and Amazon S3 authentication using pre-emptive authentication or challenge response mechanisms as applicable. The UltraESB also ships a very high performance WS-Security library that allows it to perform WS-Security operations over 3X faster than solutions based on the WSS4J/Rampart libraries used by other ESBs.

JMX support has been thoroughly improved to utilize MXBeans, and allows detailed monitoring and management of a runtime using JMX consoles such as Zabbix. The UltraESB now ships with a native Zabbix agent for even easier monitoring. As the UltraESB integrates with the Spring framework and supports Java as well as JSR 223 scripting languages, integration with third party libraries and extension is simple and powerful.

UltraESB implements clustering

You may already know from Asankha's blog or from the New item that we have released the 1.4.0 version of the UltraESB few hours ago, and I was waiting and waiting to write about this awesome clustering implementation that I have done for the UltraESB 1.4.0

The story begins with the decision to use ZooKeeper for the clustering implementation, which is what Asankha was initially suggesting me. I've done few feasibility studies and found it to be just right for the clustering implementation that I have been dreaming for years. Why I said I was dreaming is that, most of the ESB implementations out there have not identified the 2 main concerns of clustering;


  • Group coordination

  • State replication
to be different and they failed to keep those 2 concerns nicely separated. What I believe is that these 2 concerns of clustering needs to be managed separately and hence implemented with right amount of isolation yet integrating them together to work nicely. It is like spice or salt for food, you need that but only the just right amount, too less or too much gives you trouble. I have explained about this in more detail in my Clustering Part I article.

As explained in the above article UltraESB has cleanly separated the group coordination and cooperative control aspect of the clustering and that has been implemented with using ZooKeeper. On top of this we have written a command framework to command the complete cluster in one go, which enabled us to implement several set of cluster wide management controls.

First and the most important/useful cluster management command that is available on the UltraESB is the Round-Robin graceful restart of the nodes in the cluster. One of the major concerns on the modern ESB clusters was to managing the maintenance restarts without affecting the availability. I haven't come across any ESB out there including commercial once which provides a single control to do the maintenance restart with zero down-time of the service and zero message loss of the system. UltraESB guarantees this with the above operation. Restart being a graceful it guarantees that any node will not restart without serving all the messages it accepted, and the nodes in the cluster are being restarted in a round robin fashion makes sure that the system do not have a state where it doesn't have any live node to dispatch the message to, even if the cluster is just 2 nodes.

One key feature of the new 1.4.0 release is it's web based management console, which I will keep to a separate blog, but I had to mention it here to show you the one of 2 options available for you to try out this in action. So the first option is using the console Cluster Control Panel as shown below;

The other option is to use the jconsole. Which lead me to say a word on the improvements that we have done on the JMX management aspect of the ESB. Now we are using all new MXBeans and not MBeans, and that helped us a lot in improving the management. Will talk about all those on a separate article on the console and its implementation.

Apart from that this command framework enables you to turn on/off sequences, endpoints and proxy services in the complete cluster in one go. Not just that but you could write your own control commands to the UltraESB cluster too and use them in your solution to command the complete cluster.

If you are seeking to try it out the Clustered deployment of UltraESB and control commands is all what you need to go through. You may run the console using the uconsole.sh or uconsole.bat and see this in action in the console too.

That was about group coordination, and then I have evaluated several frameworks for implementing the state replication as that is a significant part of a good clustering implementation and many aspects like scalability and performance measures has to be taken into account when developing such a replication framework. We found the ehCache to be a good fitting distributed cache which can be used to implement state replication, but that used some kind of multi-casting based peer discovery to discover cache peers and we didn't want to rely on that as on cloud it is giving trouble and may not be scalable in a considerably big cluster. There we have used the right amount of salt and use the underlying ZooKeeper based coordination to discover the cache peers of distributed ehCache, thanks to ehCache it had a flexible API and the peer discovery was plugable. I am planning to write a complete article on this aspect of clustering later as the Clustering Part - II and once I am done with it, I will share the link as a comment too.

With that it brings me to the end of the story. Stay tuned to hear more about the UltraESB 1.4.0 and its newly born uconsole.

Wednesday, April 6, 2011

UltraESB v1.4.0 development builds are available for feedback!

Good cooking takes time. If you are made to wait, it is to serve you better, and to please you.
- MENU OF RESTAURANT ANTOINE. NEW ORLEANS

Anyone who has read the Mythical Man Month by Brooks will remember the famous statement on the menu of Restaurant Antoine. Likewise, we've taken a bit of time, to do a great release of the UltraESB v1.4.0, to please you better! The release is expected later this month, but we've started releasing public development builds for user feedback.

Some of the highlights of the new release are:
  • Support for clustered deployment and management of a cluster (based on Apache ZooKeeper)
  • Configuration updates, includes and removals at runtime including a "prepare" phase that detects possible issues before a configuration switch
  • An extremely lightweight Web based console for management of the UltraESB and basic monitoring
  • Complete revamp of JMX functionality to use MXBeans, and including a built-in Zabbix monitoring agent
  • Support for local and distributed caching via ehCache (expect support for memcached soon!)
  • XACML support using PicketBox XACML
  • Support for save/restore of test setup from the ToolBox
  • Many more enhancements and rectification of issues

Here is a brief screenshot from the web console


And here is a view of a proxy service shutdown command on a cluster, where one can see the list of nodes where the command completed successfully, or failed

The console also allows one to quickly load new configuration fragments, or unload or update loaded fragments


  • To checkout the source code, use Mercurial as follows:hg clone https://bitbucket.org/adroitlogic/ultraesb

Note: Some of the pages/parts of the pages of the Web console are still under construction