A recent comment by Yucheng on my preview of Dita4Hudson post finally motivated me to upload my starter code to SourceForge so that others can contribute and make use of it.

The plugin right now is mostly front-end code and does not yet run the DITA Open Toolkit transforms.  The intent of the plugin is to very simply provide text fields that a user can enter any and all applicable transform parameters for a given output type and then in the background those would be passed to the toolkit in the appropriate form and basically be a command line call.

I would love to see others pick up where I left off. I hope to be able to continue working on it again in the future, but for now I have too much else going on. I would be happy to help someone out with getting going with it. Please comment or use the contact form to get in touch.

• Dita4Hudson on SourceForge

I am progressing a bit further on my plugin that will allow for easier DITA builds within the Hudson continuous integration server.  Hudson is a quite handy build management system that offers you a lot of additional value on top of your existing builds and processes.

You can use your existing DITA-OT Ant scripts to build your output within Hudson. Many Dita users are not Ant experts and often Ant is one of the limiting factors in Dita adoption by smaller teams. I hope this plugin will help further promote DITA and provide another simple tool for Dita builds.

All feedback and suggestions are welcome.

Global properties configuration

Defining your DITA Open Toolkit installations. You can define multiple installations if it is ever necessary to use a back level version or for testing new versions prior to moving to production.

Adding a Dita build to a job

Jobs can include many steps and multiple builds. If you are building an Eclipse feature you might include your code build, context sensitive help build, and DITA transform to help topics within the same job.

The Eclipse help system is an extremely valuable tool that seems to intimidate new users.   This post is intended to cover some of the very basics about how to troubleshoot your plugins in an Eclipse help system.

Some basics first:

• A plugin is a folder or JAR file that provides instructions to Eclipse for how to process.
• Those instructions are given in the plugin.xml (extensions) and in the manifest file (prereqs, dependencies, etc).

Ensure the plugins are in the correct directory

While that might sound obvious, I have seen forum posts where the distinction between the workspace and plugins directories were causing problems. The workspace can be considered the directory where your source projects are stored for editing. That location is not processed by the Eclipse framework. You need to ensure that you store your plugins in the eclipse/plugins folder and in the correct format.

After you’ve checked that your plugins are in the correct location, verify that the contain the necessary files.

Plugins are formatted correctly and contain manifest/plugin.xml files

Open up your eclipse/plugins/my.plugin.folderorjar and check to ensure the plugin.xml is in the root of that folder or archive. You might optionally also have a META-INF/manifest.mf file.

Determine if Eclipse is picking up your plugin

Launch your help system or infocenter. If you are launching the help (Help->Help contents), you will want to open it with an external browser rather than the built-in browser. You can either change your preferences or you can just copy a TOC link address and paste it into a different browser. The port that the webserver for the Eclipse help system changes with each restart so you’ll need to determine that:

Your help URL will probably look something like http://127.0.0.1:3578/help/…. Once you have that, change the URL to http://127.0.0.1:xxxx/help/topic/my.plugin.folderorjar/plugin.xml.

If you get a topic not found or broken link

Then Eclipse did not pick up your plugin, which means something is likely wrong with your plugin.xml or manifest.

This could be a malformed XML file due to an XML markup error in your plugin.xml. Check that all tags are closed (</myclosetag>). Also, check for character encoding problems. XML is pretty strict when it comes to special characters and the most common problematic character is the ampersand (&). The ampersand in XML or XHTML (especially in links) should be represented in the file as: &amp;

Also, check that you are using the correct URL for your plugin. Eclipse uses the plugin ID in the path rather than the folder name or JAR file name.

If your plugin.xml file displays correctly

Then Eclipse found and registered your plugin. If this does not solve your problem, I am guessing you are wondering why your contents do not show up in the navigation. For your navigation file to be picked up, you have to register the TOC XML  file with the Eclipse org.eclipse.help.toc extension point in the plugin.xml.

If you want your toc to appear as a first level container, then you need to set primary=”true” for that TOC file:

<extension point=”org.eclipse.help.toc”>
<toc file=”toc.xml” primary=”true” />
</extension>

If you want this TOC file to be a child to another TOC, then you would set primary=”false” and link the maps together using a link_to (navref in Dita source) element or anchor/anchor_to (anchor & anchorref in Dita source).

You updated a plugin and the changes are not displayed

This is likely due to caching that Eclipse performs. You can either start Eclipse with the -clean parameter passed to the executable or you can manually clean the cache.

To manually clean the cache: stop Eclipse, find your Eclipse installation directory. Within the configuration subfolder, delete all of the contents except for the config.ini file and the org.eclipse.update directory.

Other issues?

Post your Eclipse help issues  as a comment and I’ll try to address them here also.

I discovered today that nearly four years after filing our initial patent paperwork that our US patent was finally issued. A few months back, we received a Chinese patent for the same submission (#ZL200710001513.4).

You can read more about the work that went into this submission and also read the patent itself.

Recently, the tool that inspired this patent was bundled as embedded feature of the IBM InfoSphere Replication Server Version 9.7 that shipped to customers this last June.

Photo of the actual Chinese patent:

Did you know that your Eclipse documentation plugins can contain Java for creating dynamic content? If you’ve ever felt limited by the Eclipse Help System, this post will show you the basics of creating a dynamic plugin by taking advantage of the org.eclipse.help.contentProducer extension point.

Skill level

I would call this an intermediate topic. Attached is a sample project that provides a simple framework for getting started with your own plugin. If you have a very basic level of Java experience or other coding experience, you should be able to follow along. I am definitely not an expert in Java either.

I know from experience that figuring out how to extend the Eclipse help system is difficult and supporting information is sparse. My goal is to give you a head start. I hope that someone out there, who is a better programmer than I, can use this information and contribute something really cool back to the community.

Getting Started

I am using Eclipse 3.5 IDE for Java EE Developers (Galileo), but you can use previous versions that contain the Plugin Development Environment (PDE). Screenshots and process might differ for other versions.

1. Install Eclipse by extracting the downloaded ZIP file and launching the Eclipse executable.
2. Download the sample Eclipse project for a dynamic documentation plugin.
3. Import the zip file as an archived project: File->Import->Existing Projects into Workspace (screenshot). Choose the Select archive file option and browse to the downloaded sample project zip file. Click Finish and you should see com.mrscripter.dynamicsample.doc listed in your Project Explorer view.

This sample provides a very basic example of producing dynamic content using Java and Eclipse extension points.

Run the sample project

Compiling and creating the output plugin in Eclipse seems a bit odd and not obvious. You can think of your sample project as a source file project that is contained within your Eclipse workspace. The compiled version of the plugin will go into your eclipse/plugins directory so that we can test it in the help system.

1. Right-click the project folder in the Project Explorer view and select Export.
2. Select Plug-in Development->Deployable plug-ins and fragments (screenshot).
3. Browse to your the directory you extracted your Eclipse download.  You should select the folder that contains your eclipse.exe file. The export wizard creates the output jar file in the plugins subfolder, for example: my_dir/eclipse/plugins/com.mrscripter.dynamicsample.doc_1.0.0.jar.
4. Restart Eclipse to register this new plugin with the help system.
5. After Eclipse restarts, click Help->Help Contents to display the help system.

You should see a top-level navigation entry named “Dynamic Documentation Sample.” Each table of contents entry in this section links to the same resource but passes in a different querystring parameter that the sample uses in the title and body.

The links might look confusing at first. I’ll walk through each of the different aspects of the plugin to explain how the plugin works. If you’re adventurous and have a good understanding of Java, you can probably stop here and just explore the source code.

If you modify the plugin code, you’ll have to re-export and restart Eclipse each time. You can run Eclipse in infocenter mode to do this more quickly and easily.

Dissecting the plugin

The plugin.xml file contains the instructions that instruct Eclipse how to process the files contained within a directory or JAR file that is in the plugins directory. These instructions take the form of extension points.

The org.eclipse.help.contentProducer extension point in the plugin.xml is the key to creating dynamic output. You add this extension point to your plugin.xml and register a content producer by specifying the name of the Java class that you want to control the dynamic processing:

<extension
   id="com.mrscripter.dynamicsample.doc"
   name="Dynamic documentation sample plugin"
   point="org.eclipse.help.contentProducer">
     <contentProducer producer="HelloDynamicWorld"/>
 </extension>

Content producer

In my sample, I point the content producer to my HelloDynamicWorld.java. This class must contain the required getInputStream method, which allows the plugin to send its content to the browser:

public InputStream getInputStream(String pluginID, String href, Locale locale)

You return an InputStream from your own Java code back to Eclipse for display. An InputStream is a byte representation of your text.

Dynamic content

Your typical HelloWorld application is not dynamic so I added a little more code to demonstrate changing the output based on some simple input.  This sample looks for a query string parameter named “title” and gets the value of the parameter and then inserts that value into a template. The result is returned as an InputStream for display.

The URL that is called is http://…../help/topic/com.mrscripter.dynamicsample.doc/something.html?title=Testing+A+Title. The content producer class intercepts all URL requests for the plugin. You can change something.html to just about anything and it will still work as long as you provide the title in the querystring.

One issue, I have not solved yet is that if you specify an extension other than .html, some browsers won’t display the result as HTML but rather as plain text.  I still need to figure out how to specify the content type in the response.

Template

I figured most people wanting to create some kind of output would likely need to do something with a template so I included an extremely basic example of variable replacement. The createMyDynamicOutput method gets the title from the querystring and then replaces the %TITLE% from the template/template.html file with the value of the query string title parameter.

What kind of useful things can be done with this?

I see these Java capabilities in most cases as a method of providing services in the help system. By services, I mean things like feeds, utilities, accessing external information, and lastly simple dynamic HTML output.   I think the following areas would provide the best return:

Transforming DITA on-the-fly

The most obvious would be to dynamically transform DITA on-the-fly and make use of the metadata to create custom documentation. You could create XML output for dynamic toc files for use in the navigation pane. If you created a front-end application using Ajax, you could allow for help system customization by tracking preferences. A user could mark that they have Product A on Windows and you could use that information to exclude information about Product B or Linux.

Widgets

You could connect to and receive Twitter feeds by integrating the Tweet4J library and display the results as a widget in your Help system homepage or in each page of your help system.  I think a lot of mashup options are possible when you consider web services that might be available to you. You could integrate other social media tools into your documentation such as integrating with your forums or otherwise create a method for commenting on your topics.

You’ve long been able to include JavaScript in your help topics, but JavaScript can only do so much. You run into cross-domain limitations when trying to use Ajax to different servers for security reasons. Java can allow you to proxy and cleanse the remote data before displaying it.  This could allow you to display RSS feed widgets in your help system. If you controlled that remote RSS feed, that might provide you a mechanism for including real time, relevant information in your help system that you might not be able to otherwise include without your product team releasing a new version of your product.

What next?

Get coding! I’m working on some projects myself to make use of what I’ve learned while putting this together. I’ll post those projects in the future. If this post or sample are useful to you, please leave a comment. If you develop a project of your own, I’d love to hear about it.

Links can be painful to deal with in help systems especially in multi-author environments or with larger numbers of topics.  Maintaining links over time can also be difficult. If you are authoring your help in DITA and using relationship tables, you might find they aren’t as easy to maintain as many claim. The Eclipse help system can make link validation difficult due to its frameset and difficulty for most link checkers to find all of your pages.

My goal in writing this is to find a link checker that meets the following requirements:

• Recursive link checking.
  Need to be able to specify how deep the program should follow links to discover and validate all links. Ideally, could limit this recursion to the same site.
• Reports that can be saved and distributed
  Share results with writing teams for resolving and fixing links.
• Customization of settings, especially links and link patterns to ignore.
  Link checkers often have false positives for various reasons and it is helpful to be able to filter and hide these “errors.”
• Bonus: Works with dynamic pages and JavaScript.
  The newest versions of the Eclipse Help System rely soley on JavaScript to generate the navigation tree. Ideally, a link checker could work with the same view that we see. Often, link checkers are just verifying the response codes in the headers rather than actually examining the page for dynamically generated content and links.
• Bonus: Command line support.
  Integration into a post-build process or for otherwise automating will likely require a command line interface.

I’ve long used ParaSoft WebKing for accessibility and link checking for projects within IBM but it can be quite difficult to configure to meet the goals above.  I’ve also tried a few other applications including LinkChecker, W3C Link Checker, and Xenu. Xenu works pretty well for discovering broken links to any kind of resources, including images, stylesheets, and JavaScripts.

Robots.txt

Many link checkers try to follow the rules  that websites define in their robots.txt file. These files instruct search engines and other spiders how they are to behave when snooping around a website. I ran into this problem when trying to test my IBM information center by using the LinkChecker from SourceForge and W3C Link Checker.

My solution was to run the link checking program against my test infocenter rather than our production server. This eliminated the W3C link checker from my possible tools because it cannot access my intranet. Xenu does not follow the robots.txt instructions.

What URL to use?

You might also assume like me that just pointing your link checker at the top level URL should work just fine. I’ve had mixed results trying to use http://publib.boulder.ibm.com/infocenter/idm/v2r2/index.jsp as my starting page.  Your link checker might also have a difficult time with the frames also.

I suggest experimenting by also running it against the TOC that is used for non-JavaScript enabled browsers, which for my IC is http://publib.boulder.ibm.com/infocenter/idm/v2r2/basic/tocView.jsp. This page does not use the standard JavaScript based navigation tree so a link checker should have an easier time with it.  If you set your link checker to recursively dig down enough levels the navigation starting point should work pretty and even better if you provide plenty of cross-linking within your files.

No broken links?

There is an Eclipse bug about certain versions not returning a proper 404 response status.  If you are experiencing this same problem, you might need to try testing with a different version of Eclipse. I’ll try to track down this bug # and put it in here.

Checking the links of my infocenter

I installed the latest version of Xenu, which had recently been updated. I pointed Xenu to http://publib.boulder.ibm.com/infocenter/idm/v2r2/index.jsp and left all of the default options.  This infocenter contains approximately 17,000 topics for all languages and are owned by some 12 or more contributing authors.

Xenu is still running but it looks as if it is properly crawling the site and finding problems:

Starting to see plenty of red showing up and quite a bit more than I was expecting so I expect I’ll be sending this report out to our team.  The total number of pages and files to test were discovered thus far is over 10k so the link checker seems to be working correctly.

When Xenu finally finished, I generated a report (Hit Cancel on the FTP screen) and it is formatted for viewing from a couple different perspectives.  The HTML markup behind the scenes is very basic, I’ll have to investigate how to parse it to show broken links by plugin for being able to send the reports to the correct people but that might be a later post.

Here’s a quick screen cap showing one easily identifiable broken link:

Other tools?

I’ve listed a few here from my own experiences but would love to hear from others.

I recently stumbled across the WinAnt Echidna project on SourceForge while browsing the DITA Open Toolkit project forums.  While DITA as a markup language is not difficult to learn and begin to start creating content, DITA can be difficult to see the fruits of those labors if you are just getting starting with the DITA-OT. Most information developers are not experts in XSL or ANT.  The WinAnt tool can help them get started with producing DITA sourced content very quickly.

WinAnt is generally going to appeal to writers and teams on smaller projects. Larger and more complex projects and deliverables likely have requirements that go beyond what WinAnt provides.

Benefits

• Free
• Quick to create and run both simple and customized builds
• Supports a variety of output types, including: XHTML, PDF, Eclipse plugin, HTML Help, JavaHelp, DocBook, and Word.
• Repeatable builds by saving build projects.
• Style settings can be controlled through skins, which list headers, footers, stylesheets and includes . Skins can be used for consistent settings amongst a team.
• Automation through scheduled tasks and saved build projects.
• Logs are styled and highlight messages produced by the build.
• ANT build.xml can be exported if your project needs to grow beyond what WinAnt provides.

Limitations

• Unclear about longevity and maintenance of the project
• Unable to customize or override XSL settings.
• Errors and warnings in build are not explicitly displayed. Must manually check logs for each transform. Expected a pop-up or dialog telling me that my project had X errors and Y warnings.
• Lack of support for specializations? (Unverified but looks that way from a quick attempt)
• Troubleshooting problems can be difficult.
• Runs on Microsoft Windows only.
• WinAnt help launches in odd sizes and is always on top of UI. No tooltips or descriptions for many settings.
• Lacks expanded tasks such as source code control extraction, distribution of output options, versioning of builds,  and advanced reporting.
• JavaHelp and HTML Help require additional installed programs that not distributed with program or linked from the UI.

When I first downloaded the product, I expected that I would be disappointed with it just based on my first impression. WinAnt for such a small and simple program is quite powerful.  I would suggest that anyone who is trying to get started learning DITA should definitely give this program a try. It definitely lowers the learning curve for DITA builds.

The WinAnt tool was created by Tony Self of HyperWrite. Nice work!

Wikis for documentation make sense for many reasons, including  low cost of implementation, ease of publishing, and collaboration possibilities.  DITA has become a popular XML format for semantic markup of information and is generally used for documentation.  Wiki content is generally authored or stored as wikitext, which is an non-standardized markup format.  Should DITA be used as a markup format for wikis instead of wikitext or HTML? I believe the answer is that DITA in the authoring environment of wikis is impractical and does not work for general audiences.

I work at IBM where DITA was first invented and where we author an unimagineable amount of content in DITA. We create all kinds of tools to support its authoring, transformation, translation, specialization, and delivery.  DITA has in various capacities been explored as a format for a variety of delivery mechanisms, including content management systems, wikis, microformatted XHTML that retains DITA markup, and also your more standard outputs.  Basically, if some tool or application is out there, we’ve likely tried to make it DITA friendly.

Know your audience

You must know your audience to create quality documentation that meets the requirements of the user.  Most of my users are database administrators and application developers. My guess is that none of the users of the products I work on knows a thing about DITA markup.  DITA requires some upfront education for anyone that is going to create content in the desired manner.

A core feature of a wiki is the collaborative editing and authoring aspect. By using DITA as the wiki markup, you are removing that feature by subjecting your potential authors to learning a new language and all its nuances. The word wiki in Hawaiin means quick. Quick also implies easy.

Subjecting one of our users to DITA authoring would be a significant hindrance in terms of their ease in offering contributions. Public collaboration on most documentation projects is extremely difficult.  Tom Johnson in his article on A Few Surprises in using a Wiki for Documentation discovered that most contributors were likely to only make small edits rather than add to the documentation set. Why throw another roadblock in the way?

Editing tools

Last time I checked, there were very few web-based DITA editors. The primary contenders were DitaStorm, XMetal XMAX, and Xopus.  I have tried both DitaStorm and Xopus for working with DITA. DitaStorm seems to be the more generally well known web-based tool of the bunch. I’ve tried it through its various versions and my general feeling is that for very basic authoring it works okay but for more specialized authoring and design it is unwieldy and often buggy. Within IBM, we had a few licenses for a project and I often found myself wishing I was just using Notepad instead for editing topics. The problems and challenges I experienced I feel were more due to Javascript issues especially around cursor placement, but as an experienced DITA user found myself very frustratred to not be able to easily get the output I desired.

Most people author their content visually to create the desired elements and effects. When they want emphasis on a term they will use bold or italics. They won’t want to consider should that emphasized text be a UIControl, parmname, apiname, varname, or one of the many other semantic tags–they will use the non-semantic bold tag.

Solutions?

Do you try to hide the complexity of authoring in DITA from users that do not care and just want to write? Do you create a kind of hybrid DITA wiki that features some of the benefits of both worlds? Is a light-weight CMS really a better solution for easy authoring? Maybe a DITA to WordPress import plugin or to Drupal or Joomla?

Currently, I don’t feel there is a good answer for the human element of DITA and wikis. I responded to Tom Johnson’s blog post when it was first posted stating that I believe there is common ground between wikis and DITA but most of that common ground is in terms of technical implementation rather than usage patterns.

I am not saying there is not a place for a native DITA wiki, but the use case is probably for collaboration among DITA authors rather than with a larger community of end users.

It always baffles me why the first place that people often try to apply new technologies is to the Kitchen, specifically recipes.  This practice reminds of starting with Hello World for programming and Lorem Ipsum Dolar for design.  Basically that says “we have some cool tools but we really don’t yet know what to do with them.” The next logical step is somehow making this tool apply to recipes right? Then you have a tool and a purpose.

The article that got me thinking about this topic is the Dropbox for iPhone Makes a Great Kitchen Aid article. I am a Dropbox user and it is quite nice for syncing files between my various machines especially project files that I might need at any time but recipes on my iPhone? Other tools handle that quite nicely already.

The other item that got me thinking about this was the Honeywell 316 “minicomputer” built in 1969 and marketed towards housewives.  A kitchen computer in 1969 now that was really cutting edge, but that housewife would also need to be quite cutting edge and know how to program the recipes in a binary format (came with a two week training course!).

There are other examples of this same practice, such as the Internet refrigerator and other networked appliances.  Is technology just trying to reach that cooking nirvana? The one where I don’t have to think about cooking and I have delicious automagically created food and served to me by a robot?

Maybe that is the key here…geeks like me love to eat, have programming skills, and rather than work on our cooking skills we’d rather try to program our way towards dinner.

I have a really hard time describing my role to people outside of IBM.  If I wanted to be lazy and did not want to go into details and explain all my different projects, I would likely quickly state that I am a technical writer.

I do not consider myself to be a technical writer though because I believe the term is so limiting. I worked as a web developer for five years while I was completing my Bachelor of Science degree in Technical Communication, which is from the department of Human Centered Design and Engineering at the University of Washington.

Most of my work experience has been in the realm of web development and my degree work was an even mix of user experience engineering and technical writing.

At IBM, while working as an information developer, I’ve spent a majority of my time working on more technical projects such as writing and developing for the IBM Data Studio Administration Console and developing a WebSphere MQ script generating tool as part of the WebSphere Replication Server team.

After that team, I moved to the Integrated Data Management portfolio as the overall Information Development infrastructure lead.  My job title does not get more clear with this change in roles. I now lead a team of database developers, writers, sales enablement, and other subject matter experts in the development of a common sample database that our portfolio of greater than 30 products. In addition to leading that development team, I am also working on education and advocating adoption of this database across other Software Group products.

That alone sounds like a full time job right? Just one part of my new role. I also am the lead for integrating all of the documentation from the portfolio of products into a single solution-oriented information center. I also develop and implement an automated DITA build system to allow all of the writing contributors to collaborate on the design of the information in an on-demand build environment.

I developed an web application that is based on the ExtJS JavaScript library, CodeIgniter PHP framework, and MySQL that powers this back-end DITA documentation build, which uses ANT, Perl, and MySQL.

I also have a good user experience engineering background from my course work at the University of Washington. In one of my most influential courses (User Interface Design), my team developed and won the award for best designed project.  I was the interface designer and technical writer for that joint team of 3 industrial engineers and one Tech Comm. student. This course was led by the world renown Dr Thomas Furness.

So what title should I really be using to describe myself information developer? software engineer? application developer? user experience engineer? team lead? technical writer? or just nerd?