I joined IBM as an information developer intern (co-op) in June of 2005.  Dell Burner, my team lead, was working on a project with our visual designer and user experience engineer to create a quick start visual for setting up WebSphere MQ for use with Q replication. Q replication is a database replication technology that is extremely high performing due to its architecture and infrastructure. The integral piece of making Q replication out perform standard SQL-based replication is that it pushes its instructions over WebSphere MQ message queues.

WebSphere MQ administrators are a whole different audience than DB2 database administrators.  These two different products are like night and day. DB2 database administrators typically don’t know anything about WebSphere MQ and also are not likely allowed to touch the WebSphere MQ configurations.

Q replication was facing a major customer pain point when it came to configuring WebSphere MQ. The DBAs were not skilled in the technologies needed to implement it. In most shops, there were communication and procedural barriers to getting the environment configured. In most cases, the best case scenario for getting just the WebSphere MQ portion of the environment configured was a minimum of 1-2 days. If the DBAs were working in a z/OS shop, that time frame could be even longer.

Of course a best case scenario means all parties involved are well versed in their requirements and can communicate those from DBA to WebSphere MQ admin.

In an effort to ease the learning curve of Q replication, Dell Burner, Daina Pupons-Wickham, Kevin Lau, and Michael Mao undertook an effort to create a diagram based document and supporting instructions for assisting DBAs to create their setup scripts that they would ask the WebSphere MQ admins to run.

My contribution

I was just two months into my coop and Dell was showing me the PDF of the diagram that he was going to present to one of our vice presidents later that day.  I had worked with PDFs a bit in previous jobs and at school and knew that I could add some additional functionality to his design that would improve it further.

In its current state, the diagram was intended to be printed out and the DBA was expected to hand write on it the names of the objects (while cross referencing the docs) that they expected their WebSphere MQ admin to create.  That first day before Dell presented to our VP, I was able to add text input fields that a user could type into instead of hand writing. I also added tooltips to each of the fields that added embedded assistance to help with explaining each of the objects to cut down on the need for looking up documentation.

Dell’s presentation was a hit even though it was just minor updates to the existing diagram.  As much as I added in just a single day, I knew there was still significant room for improvement.

PDFs support what is called “calculations” and other processing of form entries. An author can add JavaScript instructions and functions into the PDF that allow it to become more dynamic. I had already added all of the text fields and tooltips–why should we expect the DBAs to assemble their setup scripts when we could do it form them?

I created additional scripts that would take a users input into the diagram and transform the input into fully functional WebSphere MQ scripts that could be run as is. The instructions and PDF forms were also customized depending on the type of Q replication environment (unidirectional, bidirectional, peer-to-peer, event publishing, etc).

Download a copy of this tool from: WebSphere MQ Script generator for Q replication and event publishing

Watch a demo of the tool at: The MQ Script Generator – ChannelDB2

User testing

While still in the draft state and testing the scripts, we performed user testing and validation. We were able to measure that the new PDF tool was able to reduce user task time from 24-48 hours to down to 15-20 minutes.   Obviously a significant improvement in time and thus customer’s ROI.

Patent application

After some thought and later research, we determined that no one else had done anything similar. We filed our international patent application (US Patent App # 20070162891) in January of 2006. In July of 2009, we finally heard back from our first patent office–China granted our patent for our invention “Graphical aid for generating object setup scripts.” Chinese patent number is #ZL200710001513.4.

Later additions and enhancements

Eventually Adobe added the capabilities to Adobe Acrobat Professional that allowed content creaters to create “rights-enabled” PDFs. This basically means you can allow a user of your PDF that only has Adobe Reader to be able to save their filled out forms. Previously, the lack of that capability was extremely limiting.  With the addition of this capability, the InfoSphere Replication Server product now bundles my tool as a launchable function from the GUI.

While this PDF definitely helped reduce a major customer pain point, I believe its implementation could have been done in a richer way if we could have gotten a programmer to build the actual functionality into the product. This began as kind of a demonstration and turned into its own full-fledged tool.

Other use cases of this capability are constantly popping up so you might see something come along soon.

Install this script from userscripts.org: IBM Lotus Connections Drop-down menus

IBM’s Lotus Connections software is an enterprise social software application that consolidates a lot of collaboration and information sharing within the enterprise to a single location. We use Lotus Connections quite extensively within IBM and it has been an amazing improvement over many of our previous tools and methods for project, team, and professional collaboration.

The recently announced Lotus Connections 2.5 features communities, blogs, wikis, file sharing, social bookmarking, activities,  profiles, and a customizable dashboard.  I can tell you that I use every single one of those features on a daily basis and its been a great step forward in so many ways. Knowledge is being better shared across our enterprise and it can be found more easily. Projects and teams can stay organized and on track. To-dos can be assigned and tracked.

While my overall satisfaction with the product is quite high, one small design issue bugs me. Thankfully, Lotus Connections is a web application that I can use the Greasemonkey addon for Firefox to alter aspects of the design.  I often find myself having to switch between sections in Lotus Connections–On a wiki page and need browse to a wiki section within a community.  The default design has a top bar menu with only top-most navigation items:

Getting around Lotus Connections sometimes requires clicking through multiple unnecessary pages.  I decided to add drop-down menus to each menu item to reduce the number of clicks to get around. I eventually discovered the ATOM feeds for each Lotus Connections area and determined I could add links for specific areas such as a list of all your communities or a list of all files that have been shared with you (had to blur out the actual files in the screenshot):

Overall, this script helps improve an already awesome application a bit more.  I’m hoping to convince the Lotus Connections teams to implement some kind of navigation that either matches this experience or something similar for being able to more quickly navigate to specific items in your network.

Using this script

You can use this script on http://ibm.com/communities, which is an IBM implementation of Lotus Connections 2.5. You can also customize the locations that this Greasemonkey script runs to include your own installations. For example, if you want it to run against http://mybusiness.com/connections/* you would include that location:

In May of 2008, I switched roles at IBM. I moved from being your standard information developer (technical writer) to taking on a much more open ended role that I often have a difficult time describing, internally titled “Information Development Infrastructure Lead” for the IBM Integrated Data Management portfolio of products.

One of the facets of this new, broad role was leading a team of developers, tech sales, information developers, and subject matter experts in the creation or adoption of a common sample database for the entire portfolio. The IBM Integrated Data Management portfolio is mostly comprised of IBM Optim products and IBM Data Studio and some other tools thrown in. Overall, the portfolio has potentially 30 or so products with their own database requirements.

Adopt or create?

IBM has a variety of sample databases around the organization. Some are small, some are random, some for very specific situations, and very few are realistic or rich for a broad use.  My first task was evaluating all of the potential sample data sets that were available to our teams and to try to find one that best fit our requirements.

Key requirements

The outcome of the project needed to meet some key requirements for our products:

• Story or scenario that drives usage and design of the database.  Must be be similar to a realistic customer environment to help relate our tools and information to solving the customers’ own problems.
• Support the wide variety of databases that our tools support, including DB2 for Linux, UNIX, and Windows, DB2 for z/OS, Oracle, Informix Dynamic Server, and Microsoft SQL Server.
• Support key product scenarios and features, including data warehousing, data mining, modelling, application development, and general database tasks.
• Support XML capabilities in the various databases.
• Support spatial data teams (mapping data).

Very few of the databases my team investigated came close to meeting any of these goals.

Cognos acquisition

Earlier in the year, IBM had acquired the Business Intelligence software company Cognos. Cognos brought to IBM an extremely rich, realistic, relevant, and widely adopted sample database known as the Great Outdoors sample database (GSDB).  This sample database is based on the operations of a fictional outdoor distributor known as the Great Outdoors company.  The data set included 5 schemas: business-to-business sales data (GOSALES), warehouse data (GOSALESDW), human resource data (GOSALESHR),  retailer data (GOSALESRT), and marketing data (GOSALESMR).

This database was by far the closest match to our portfolios requirements. The database prior to our work already had the following features:

• Based on a story about a fictional business.
• Five schemas that covered different types of business data.
• Localized and translated.
  • All text within the database is available in 25 languages.
  • Data is spread out across the world and can expose different scenarios in different regions and countries.
• Initial support for DB2 for Linux, UNIX, and Windows; Oracle; and Microsoft SQL Server.

Expanding the GSDB database

To meet our key requirements, we required some additions to the sample database.  I had to carefully navigate internal political issues with this cross-organizational project and be careful with the acquisition teams to ensure that all interested parties were happy and the relationships continued to be beneficial for all parties.

Working with the Cognos teams, we prioritized and planned for the addition of a new schema and addition to the story that the database follows. To meet the needs of our sales enablement teams and also our data mining teams, we required business-to-consumer data.  This required the addition of customers as individuals rather than businesses, transactions, customer interests, fake credit data, and a variety of other information.

The  data mining and data warehousing teams had the most challenging to implement requirements of any team.  The data that we added needed to be quite deep as well as intelligent. To demonstrate the data mining tools we needed to add associations, patterns, time series, and trending to the B2C transaction data. Before the transactions could be generated, we had to first add approximately 30k new customer records.

Adding customers

The customers that we needed to add involved more manual work than some of the other aspects of the expansion of the GSDB database. We involved teams and individuals around IBM to help assist with the data collection.  Eventually over 50 people across many countries had helped with adding the following types of data:

• First and last names that were representative of the population of a country and culture. For example, some cultures have different surnames patterns for females such as Czech female surnames that end in “ova.”
• Translated or localized version of all data. For example, city names in their English  name and its local name , such as Bangkok and in Thai กรุงเทพมหานคร.
• A variety of city names, state/province names, street names, and corresponding postal codes and phone codes.

All of the above data was gathered and then randomly mixed up so that we could generate as additional rows by combinations of first names, last names, addresses and cities. Also, we did this to ensure that all data was random and that any similarities to a real personal were entirely by chance. Additional data was added in to these records randomly such as age, profession, marital status, and hobbies.

Generating complex transactions

One of the members of my extended team was a developer and subject matter expert that worked with the data warehousing and data mining teams. He was able to create an application for us that would read our new customers table and generate orders either randomly or using some logic based on age, location, profession, marital status, and hobbies.

The application read the approximately 30k customer records and generated a half million transactions for those customers. Patterns, associations, trends, and other intricacies were added to these transactions so that our complicated mining tools could discover and expose to demonstrate the value of the tools and how to use them.

GSDB sample database usage at IBM

Our new version of the sample database is starting to gain wide usage across IBM’s Software Group. Many teams are writing developerWorks articles, tutorials, and demos based on the story behind the data and the data itself. You can see many of those using the following query: http://www.ibm.com/developerworks/search/searchResults.jsp?searchType=1&searchSite=dW&searchScope=dW&query=GSDB

One of the first projects to use the sample database was the IBM InfoSphere Warehouse information development team, which created three tutorials around the sample: SQL warehousing, cubing services, and data mining.

Of course, the Cognos products also continue to make extensive use of the data within their product line and by a variety of teams including information development, quality assurance, education, and sales.