Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • New on Github: Domino JNA - Cross-platform access to IBM Notes/Domino C API methods from Java

    Karsten Lehmann  8 April 2016 19:11:37
    I would like to introduce you to a pet project that I have been working on for the last months:

    Domino JNA -

    Cross-platform access to IBM Notes/Domino C API methods from Java

    As you might have already read in this blog in the past, we have been playing with the Domino C API for some time now and found out that you can speed up view reading dramatically when you talk directly to the C API instead of using IBM's Java classes.

    The C API also provides lots of other functionality for which no Java or LotusScript methods exist yet.

    It has always been possible to build your own DLL with C code calling Domino methods and use it via Java's Native Interface (JNI). We did this in applications developed for customers and partners, e.g. to render Richtext to HTML format, directly stream attachments (IBM's API first stores files temporarily to disk, not a good idea) or replicate databases with a progress callback.

    But the problem has always been that the native code has to be compiled for each platform.

    JNA to the rescue

    Java Native Access is a project that gives Java developers access to native shared libraries without creating their own DLL or library file.
    You simply call Java methods and make sure that your method arguments match those on the C side.

    So I created my first small sample application months ago calling some Domino code, found out that it worked in Windows 32 bit, found out that I had to change something for Windows 64 bit to get more than just crashes, found out that the whole stuff did nothing than crashing in Linux systems, read lots of Domino C API documentation, build testcases, documented the API methods....

    ...and months later I uploaded the whole stuff to Github. :-)

    Here it is:

    An API is nothing without documentation and sample code. And I am working on that, but this takes time. There are already a few testcases available for the view functions.

    Since talking to low level functions is not what everybody likes, I am also creating helper functions that make working with the API easier. I rename stuff, I add stuff and hide stuff. So please be aware that this project is definitely not finish and will change, but I want to share the code today because it already creates value and works quite well for us.

    What does it do already?

    Here are some of the things you can do with the first version:
    • view lookups using different key formats (e.g. strings, numbers, dates, date ranges) and equality/inequality searches (e.g. find less or greater than a search key)
    • decodes all available types of view column data types (string, string list, number, number list, datetime, datetime list) and row data (e.g. just note id, UNID, counts, unread flag etc.)
    • read view data as another Notes user
    • separation of Notes views and their data into multiple databases (like programmatically creating private views and keeping them up-to-date incrementally)
    • special optimized functions for local databases (e.g. when API is running on the server and databases are on the server as well):
      • dynamic filtering of view rows based on a Note id list with paging support (this really rocks!)
      • reading categorized views with expanded/collapsed entries and min/max level
    • read design collection with design elements in a database
    • support for view resorting (changing the collation in C API terms)
    • fulltext index creation with all available options
    • supports incremental synchronization of Domino databases
      by reading noteid lists of modified and deleted documents (IBM's Java API does not return ids of deleted docs)
    • clearing the replication history
    • compute @Usernameslist values for any Notes user


    The Github repository contains a Maven project with the API code and setup instructions. All is available under Apache 2.0 license.

    As next step, I want to add the code to an XPages Extension Library plugin to make it easier to use from XPages code.

    Please provide feedback if the project is working for you.

    Updated on OpenNTF: Release 1.1 of Open Eclipse Update Site with Mac 64 Bit Client support and other cool stuff

    Karsten Lehmann  4 March 2016 01:23:48
    I have updated the OpenNTF project "Open Eclipse Update Site" with release version 1.1 and added the following useful features:
    • added support for the Mac Notes Client with 64 bit
    • new view action to extract selected features as an update site to disk (also available in headless mode via Java agents “(API)” and “(Java-API)“)
    • new view actions to install/uninstall selected features via rcplauncher based deployment, which is used by software distribution systems and should also work on clients where widget deployment is blocked via policy
    • new database icon

    Update: I had to renew the download package on 4th March 2016 and fixed/added the following items:
    • fixed bug where the rcplauncher deployment log could not be written for Notes data directories containing whitespaces
    • added alternative UI which can be activated via role [WidgetUILaunch]. Setting the role displays an install wizard on database open instead of the classic Update Site UI views. The wizard guides the user through a a widget (extension.xml) based plugin installation

    Image:Updated on OpenNTF: Release 1.1 of Open Eclipse Update Site with Mac 64 Bit Client support and other cool stuff

    Image:Updated on OpenNTF: Release 1.1 of Open Eclipse Update Site with Mac 64 Bit Client support and other cool stuff

    Image:Updated on OpenNTF: Release 1.1 of Open Eclipse Update Site with Mac 64 Bit Client support and other cool stuff

    Image:Updated on OpenNTF: Release 1.1 of Open Eclipse Update Site with Mac 64 Bit Client support and other cool stuff

    Notes 9.0.1 64 bit for Mac OS X 10.11 El Capitan now available - warning our customers NOT to update for 30+ days!

    Karsten Lehmann  29 September 2015 10:30:06
    One day before the general availability of Mac OS X 10.11 (El Capitan), IBM released an update for their IBM Notes Client 9.0.1 with 64 bit support:

    Previous versions were using Java 1.6 32 Bit under the hood, which was maintained by Apple for the last 7 years. Since Apple dropped support for Java 1.6 in OS X 10.11, IBM had to update their IBM Mac Notes Client to work with newer JVMs (1.7 and 1.8 are maintained by Oracle and are only available with 64 Bit).

    Unfortunately we now have to warn our customers not to update their OS X version and Notes environment for at least 30 days.

    The reason is that IBM will release the C API toolkit 30 days after the Notes Client availability, as mentioned in this Q&A page:

    NOTE: If you are running applications that make use of the Notes C API Toolkit, then you must recompile those application with the new 64-bit Mac API Toolkit, which is expected to be available approximately 30 days after Notes 9.0.1 64-bit ships. This technote will be updated when the C API Toolkit is available.

    Since the applications that we maintain on OS X for customers (such as MindPlan or individual Notes Client add ons) contain native code, they need to be recompiled for 64 Bit, otherwise they will no longer work after the OS X update.

    We are asking IBM dev for a beta version of the C API toolkit to speed up this application update process, but have not received a positive response yet.

    BTW: It looks like there are no plans to move the Windows Notes Client to 64 bit. However, for Notes 9.0.2 it is planned to ship it with IBM's JVM version 1.8. IBM will announce their release plan for 9.0.2 in January at IBM Connect.

    Looks like there is Java 6 support in El Capitan and the old 32 bit Notes Client is still working. See the comments for details.

    New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    Karsten Lehmann  6 March 2015 14:48:54
    Once again I am digging into the Notes/Domino C API to implement fast and powerful view lookups. What caused it this time was a customer requirement for "Notes Client style keyboard navigation" in web views, meaning that you type a character, press enter and the cursor position should be moved to the relevant area of the view (e.g. where the content in the first sorted view column is greater or equal letter "L").
    Using standard Java APIs for this purpose (ViewNavigator.getNextSibling(), until we find a greater column value) was much to slow, since the view had about 40,000 entries in the top level.

    We did a lot of experiments in this area in the past (1, 2) writing custom DLLs for Windows/.so libraries for Linux, but this time my plan was to use Java Native Access (JNA) from an Eclipse plugin to build the code more quickly.

    I will try to publish some tool libraries in this area in the near future (as always I am very busy...). But in this post, I just want to document a method that is available in the C API since 9.0 and has not been documented in the C API tookit since then.

    The method is called NIFFindByKeyExtended2 (the C API release notes say NIFFindByKeyExtented2) and is only mentioned a few times on the web, e.g. in Scott Souder's C API toolkit announcement post.

    Here is the method signature:

    STATUS far PASCAL NIFFindByKeyExtended2 (HCOLLECTION hCollection,
                    void *KeyBuffer, DWORD FindFlags,
                    DWORD ReturnFlags,
                    COLLECTIONPOSITION *retIndexPos,
                    DWORD *retNumMatches,
                    WORD *retSignalFlags,
                    DHANDLE *rethBuffer,
                    DWORD *retSequence)

    I asked IBM dev about the method and got the information that it combines NIFFindByKey and NIFReadEntries in one call. In earlier Domino releases, you had to use NIFFindByKey to get the view entry position for a search key and then use NIFReadEntries to read the view data. Unfortunately the view could have changed in the meantime so that the postion could point to a different location.

    This is not the case for NIFFindByKeyExtended2, which is executed atomically with a read lock on the view. For remote calls, the returned data can have up to 64 KB of content. If more data is required, subsequent calls to NIFReadEntries have to be made. For local calls, the main use case for XPages apps, the buffer has no limit.

    Here is the complete method documentation:

    /*        NIFFindByKeyExtended2 - Lookup index entry by "key"
    *        Given a "key" buffer in the standard format of a summary buffer,
    *        locate the entry which matches the given key(s).  Supply as many
    *        "key" summary items as required to correspond to the way the index
    *        collates, in order to uniquely find an entry.
    *        If multiple index entries match the specified key (especially if
    *        not enough key items were specified), then the index position of
    *        the FIRST matching entry is returned ("first" is defined by the
    *        entry which collates before all others in the collated index).
    *        Note that the more explicitly an entry can be specified (by
    *        specifying as many keys as possible), then the faster the lookup
    *        can be performed, since the "key" lookup is very fast, but a
    *        sequential search is performed to locate the "first" entry when
    *        multiple entries match.
    *        This routine can only be used when dealing with notes that do not
    *        have multiple permutations, and cannot be used to locate response
    *        notes.
    * Inputs:
    *        hCollection = Handle of per-user collection context
    *        KeyBuffer = Address of "key" summary buffer
    *        FindFlags = Find flags word (FIND_xxx)
    *        ReturnMask = Mask specifying what information is to be returned on each entry
    *                                        (READ_MASK_xxx).  Information is always returned in the
    *                                        order of the bits which are 1, from bit 0 to bit 15.
    *        retIndexPos = Place to receive index position of (first) matching entry
    *        retNumMatches = Place to receive # entries which match (optional)
    *        retSignalFlags = Place to receive extra sharing warning flags (optional)
    * Outputs:
    *        (Routine) = Error status (ERR_NOT_FOUND if none found at all)
    *        *retIndexPos = Index position of (first) matching index entry.
    *        *retNumMatches = # matching entries
    *        *rethBuffer = Handle of return buffer (may be NULLHANDLE)
    *        *retBufferLength = Length of return buffer
    *        *retSequence = Index Modified Sequence number

    Based on searches on the internet, there also seems to be NIFFindByKeyExtended3, but I don't have information about parameters and purpose for this method yet.

    Tools we use to build web apps

    Karsten Lehmann  21 September 2014 21:50:48
    In a recent comment in this blog, Andrew Magerman asked what frameworks I use to build web applications and whether we have looked into Angular. Since the answer got too long for a simple comment and might be interesting for others, I created this blog article.

    Server side
    I am using my own regexp based templating system, which simply fills some placeholders in static HTML and JS files and sends them to the browser, e.g. to compute URLs, to insert translated phrases and include content of other template files. I don't need much flexibility for this template system, because all dynamic stuff is created in the browser anyway.

    Data is provided via custom REST services implemented in Java that offer CRUD operations. In many cases, the server side application code uses framework classes from OSGi plugins that build a common layer shared between multiple web applications.

    Frontend stuff
    For the frontend, I use Bootstrap to build responsive user interfaces that work on phones, tables and desktop browsers. I use jQuery, lots of helper projects like bootstrap datepicker and timepicker, select2 for flexible comboboxes, velocity.js for animations, Asynquence to work with callbacks and other libraries that I tweeted about in the past.

    To wire them together I use require.js, which gives me a clean dependency resolution.

    React based UI rendering
    For more and more UI components, I use React for the rendering and their flux application architecture.
    I especially like their approach to reduce DOM changes by comparing old and new in a virtual DOM tree in memory when data changes.

    Another advantage of React.JS is that the initial UI code can be precomputed on the server side to improve SEO ranking (see this article).

    Normally, this is done in code running in a Node.JS server.

    For Java based server environments like IBM Domino, we have code to do this using the JavaScript engine of the JVM and as an alternative (with better performance thanks to Node's V8 JavaScript engine) by leveraging an external Node.JS process on the same or a different machine.

    We haven't used Angular in real-life projects yet. Looking at the documentation, it feels as if they over-standardized web application development a bit, making it more difficult than necessary to get started.

    What concerns me is that their templating system does not seem to optimize DOM operations as good as React does. In the browser, it's all about responsiveness and performance.

    What I don't use
    Personally I prefer not to use JSF components or similar concepts for the web UI. I don't like to depend on a server state and don't like to have too much communication between frontend and backend.

    Instead I like to have most of the UI code in the browser so that the app is responsive even with bad network coverage. In addition, I can optimize DOM rendering and have more ways to play with animations and transitions.

    Communication with the server side is done through REST APIs that can also be reused to test functionality, automate tasks or for automatic data imports/updates.

    You may have noticed that not much of my way to develop web applications requires a Domino server.
    Having Domino on the server is a great thing, as it includes many services in one consistent platform, like the document database, fulltext search, directory services, replication and a close integration of application code and the mail server.

    But there's more than one way to skin a cat. We have prototypes in the lab that don't have any Domino dependencies.

    Since we like document databases combined with replication, we have been playing with the CouchDB eco system for some time:
    the Apache CouchDB project that is currently merged with the Cloudant database and the rcouch project as well as its mobile version Couchbase Lite and PouchDB in the browser.

    Combined with an OSGi based server platform, that looks like a powerful and extensible app dev environment.

      Status report / collection of web and mobile development frameworks and tools

      Karsten Lehmann  27 June 2014 09:49:20
      The last post in this blog has been written 6 months ago. Although I have had several ideas for new posts, project work and family life got in the way (our son was born last September).

      At Mindoo, we have been incredibly busy working on development projects and product prototypes. Some of them are still IBM Domino based with Responsive Web Design using Bootstrap and jQuery or Sencha's ExtJS, others are pure JSF applications using the Primefaces framework.

      In other projects, we produced EPub files from IBM Domino data (we used EPublib for this purpose), built some native extensions to call Domino C API functions from Java code (e.g. direct attachment streaming without extracting files to disk first) and dived a bit into the mobile development space with apps developed with Appcelerator Titanium and a CouchDB on the server side as well as Couchbase lite in the mobile client to easily sync data between devices.

      Since I could not find the time to write blog articles, I more and more have been using Twitter to publish interesting frameworks and tools. I do this primarily for myself to be able to find them later when I need a tool for a project, but hey, my Twitter account is open, so feel free to take a look or become a follower.

      To be able to search my findings, I created a Notes database on our web server where I download my tweets, add content of linked web pages and use Domino's powerful fulltext search engine for searching.
      That database currently only has an ugly Notes Client based user interface, but I plan to add a simple web frontend to it. We'll see if and when this will be available.

      So for now, all I can do is recommend taking a look at my Twitter account to see what technology I am currently working with:

      Now on OpenNTF: Open Eclipse Update Site - based on IBM’s template but with extended functionality

      Karsten Lehmann  6 December 2013 19:16:01
      I just created a new project on OpenNTF called "Open Eclipse Update Site".

      The Open Eclipse Update Site database is based on the OpenNTF project "Eclipse Update Site (updatesite.ntf)" from IBM with additional functionality, e.g.
      • Mac Notes Client support (no SWT error messages like in the original template)
      • View action to delete selected features from the database (no need to delete all like in the original template)
      • Support for headless builds (automatic generation of update site, e.g. in a Jenkins server):
        database contains an agent called "(API)" that can be called to delete all content and import an update site from the local disk

      Now on OpenNTF: Mindoo XPages2Eclipse - Eclipse APIs for XPiNC applications!

      Karsten Lehmann  28 November 2013 19:45:49
      I just created a new project on OpenNTF: Mindoo XPages2Eclipse.
      Our toolkit, which provides extensive Eclipse APIs to XPages developers in the IBM Notes Client (XPiNC),

      is now available for free!

      Here is the project description:

      XPages2Eclipse is a language extension for XPages-development within the Lotus Notes Client

      Find out how XPages2Eclipse simplifies the development of XPages-applications for the IBM Lotus Notes Client considerably. With the help of this extensive toolkit you will be able to develop applications, which exhaust the full potential of the local client.

      XPages is the new technology of the hour for the notes/domino platform. It makes possible the development of modern and attractive applications in an up-to-date integrated development environment - the Domino Designer.

      For newcomers it is often difficult to master the XPages technology due to its extensive set of features - the change from classical Notes development with LotusScript is not to be accomplished within a couple of days.
      In addition to getting familiar with the usual Web standards like HTML, CSS and JavaScript, one also has to get to know the ui-library Dojo, as well as IBM specific additions like server side JavaScript (SSJS), themes or components from the Domino Extension Library.

      Yet, the result at the end of this steep learning curve is quite impressive:
      dynamic web user interfaces that can join data of multiple Lotus Notes databases or other data sources, if needed, or even an application for mobile devices – no traces left of the antiquated user interfaces that used to be created with classic Lotus Notes development.

      Thanks to the Lotus Notes Standard Client XPages applications can also be used locally and even offline.

      Expanding the boundaries of local XPages applications

      Unfortunately - from the perspective of developers - it is difficult or even impossible, to offer a set of features users are familiar with when executing XPages applications locally. That is, if developers stay within the boundaries of the XPages standard.
      There are hardly any standard APIs available to interact with the Lotus Notes Client or any other locally installed software.

      Missing are for example:

      • Integration of existing Notes applications: filling in Notes forms and Emails with data from XPages applications, accessing documents selected in classical Notes views, running existing LotusScript code
      • Import or export of data from IBM Lotus Symphony , supporting documents, spreadsheets and presentations
      • Executing document attachments with associated desktop-applications (for Windows, Linux and Mac OS)
      • Accessing the clipboard to store HTML, text, images or files
      • Executing long-running operations in the background, displaying their progress and cancelling the operation if necessary
      • Convenient features like file selection, including multi-selection, and folder selection
      But it doesn't have to be this way - XPages2Eclipse comes to your rescue. It enables developers to use functions of the Rich Client, without the need to become experts in either plugin- or Java development.
      Every feature of XPages2Eclipse can be used from within server side JavaScript.

      XPages2Eclipse offers something for everyone

      The requirements listed above are all met by XPages2Eclipse - and more!
      Your users will experience a real Rich Client-feeling for XPages applications within the IBM Lotus Notes Standard Client.
      As a developer you will save time and will be spared a lot of hard work to implement these functions, while enhancing consumer acceptance of your product.

      More information and sample code

      You can find lots of sample code and API documentation in the XPages2Eclipse wiki on the XPages2Eclipse website.

      New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes

      Karsten Lehmann  9 September 2013 22:06:28
      There is a new project on OpenNTF that I created a few days ago: Mindoo Xulrunner Prefs.js Management Plugin.
      It's nothing big, only a small Eclipse plugin that can be installed in the Notes Client to manage the preferences of the Xulrunner engine that renders XPages in the Notes Client (XPiNC).

      The main purpose for this is to set the property "dom.allow_scripts_to_close_windows" to false on a number of machines (the plugin can be deployed via policy). This enables XPages applications to close their own tab in client-side JavaScript (CSJS), something that is not possible by default yet (last tested version: Notes Client R9).

      But the even more interesting part, at least for Eclipse plugin developers, is that the project demonstrates how to run code before and after the password prompt of IBM Notes.
      We use the following Extension Point:

      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes?xml version="1.0" encoding="UTF-8"?Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes?eclipse version="3.4"?Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changespluginImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
         <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changesextension
               point="com.ibm.rcp.lifecycle.application.startBundles"Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
            <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changesapplication
                  id="com.ibm.rcp.personality.framework.RCPApplication"Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
               <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changesbundle
                     id="com.mindoo.xpinc.changeprefs"Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
               <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes/bundleImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
            <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes/applicationImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
         <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes/extensionImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes/pluginImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>

      The plugin code will actually get started a bit too early for our specific use case, because the user still has to enter his password, so no Notes session is available yet to read the Notes.ini content (or even find out where the Notes.ini is located).

      So we register an ILoginContextEventListener to get notified as soon as the user has logged in:

      public void start(BundleContext bundleContext) throws Exception {
              XPiNCChangePrefsActivator.context = bundleContext;
              //we register a ILoginContextEventListener to get notified when the
              //user has logged into the platform
              ILoginContextService service = SecurePlatform.getLoginContext();
              service.addListener(new ILoginContextEventListener() {
                      boolean hasTweakedPrefs = false;
                      public void handleLogin(LoginContextEvent event) {
                      if (event.type == LoginContextEvent.MethodEnd && !event.hasException) {

                              synchronized (XPiNCChangePrefsActivator.class) {
                                      if (!hasTweakedPrefs) {
                                              //we use a flag here, because the
                                              //method is called twice

                  public void handleLogout(LoginContextEvent event) {  }

      This technique can also be used to detect Notes ID changes and is inspired by a blog article of Hunter Medney.

      Please note that there is another Extension Point in the Eclipse platform to launch code on startup, which is called after the user has logged in:

      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changesextension point="org.eclipse.ui.startup"Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
          <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changesstartup class="com.mindoo.startuptest.MyStartupHandler" /Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>
      <Image:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes/extensionImage:New on OpenNTF: Plugin to close XPiNC applications from CSJS code / to detect Notes ID changes>

      But the Extension Point we used for the plugin seems to be called a bit earlier, which reduces the risk that any sidebar panel or open tab that uses Xulrunner is opened before, which would prevent us from changing the prefs.js content permanently.

      New on OpenNTF: Geospatial indexing for IBM Notes/Domino data

      Karsten Lehmann  30 July 2013 23:11:37
      Last weekend I created a new project on OpenNTF.org, which is part of a pretty big "pet project" that I have been working on for several month and that will hopefully be ready for primetime someday.

      My original plan was to submit this pet project for the last XPages development contest, either in addition to or instead of the Mindoo FTP Server, but the project got bigger and bigger over time - and an FTP server was finally easier to polish and explain than my other idea.

      This idea has to do with alternative indexing techniques for IBM Notes/Domino data, something like "Notes Views on steroids":
      Building an external indexer for IBM Notes/Domino that is more powerful than classic Notes Views, but still easy to use and scalable for large amounts of data.

      And while I was investigating different open source indexers and database engines, I once again came across the topic "Geospatial Indexing", which I had already discussed in the article XPages series #14: Using MongoDB’s geo-spatial indexing in XPages apps

      Geospatial indexing basically solves the task to find locations stored in a database that are close to a given set of coordinates, specified as latitude/longitude pair and to sort the results by distance.
      With all those smartphones out there that carry a GPS chip, the requirement nowadays is pretty often to "find the next Italian restaurant" or "find friends nearby" that all can be solved with Geospatial Indexing.

      In my XPages series article I demonstrated how to use an external MongoDB database to do these kind of searches from XPages applications, but this stuff gets even more interesting and realistic if we can solve it with pure Notes/Domino technologies - and it is possible.

      There are a few obvious ways how Geospatial searches could be implemented with Notes/Domino APIs, e.g. Database.search(String), fulltext searching or just manually scanning through all view entries to find the relevant documents.
      The main problem is, that they either do not scale very well, because all documents of a database have to be scanned or they require the creation of a fulltext index, which I personally try to avoid for this kind of lookups (takes a lot of disk space, is often not up to date, sometimes issues with date searches, when Domino thinks a field is not a date/time, but a text).

      The solution: Geohashes

      After a few hours of searching, I found a document that explains how MongoDB has implemented Geospatial Indexes.
      They convert latitude/longitude pairs to a single string value, a so called Geohash.

      This way, a single prefix lookup is enough to search for both values. All you have to do is to compute the list of Geohash boxes that intersect the search area and find view entries that start with the right Geohash prefix:

      Image:New on OpenNTF: Geospatial indexing for IBM Notes/Domino data
      (screenshot taken from the Geohash demonstrator website)

      Mindoo Geohash Demo

      The new project on OpenNTF that demonstrates the Geohash technique is called "Mindoo Geohash Demo" and it looks like this:

      Image:New on OpenNTF: Geospatial indexing for IBM Notes/Domino data

      Project description

      The sample database can be used to store and search real-world locations. A location document consists of a name, a type (e.g. "Restaurant" or "Supermarket"), address information with street/zip/city/country and a field for other custom data.

      When entered via the web interface, we use the Google Geocoding API  to retrieve geo coordinates (latitude/longitude) for the address.
      These coordinates are stored alongside the other location data in the database.
      Location documents can also be created via a REST API call.

      Image:New on OpenNTF: Geospatial indexing for IBM Notes/Domino data

      The database also provides search functionality via web UI and REST API to quickly find the nearest locations for a given point (either entered as address or latitude/longitude pair), sorted in ascending distance.

      To get started, simply sign the database, copy it to your IBM Domino R9 server and open it in a browser.
      The database contains a sample dataset (all Starbucks stores in New York and Berlin, all Apple Stores in Germany) as a starting point, but this data can be deleted to start from scratch.
      To search for locations, enter an address (e.g. "Brandenburger Tor, Berlin, Germany") and the maximum distance in meters (e.g. 1000) in the search form and click the search button.

      You can further restrict the result set by specifying a location type (e.g. "Coffee"). Just select a type and leave the address field empty to see all locations with that type in the database.

      Image:New on OpenNTF: Geospatial indexing for IBM Notes/Domino data

      For a visual representation of the search results, select up to 25 rows in the result list and they will get displayed via the Google Maps API.

      Hope you like the demo! All code and required libraries are available under Apache 2.0 license.