The Remembrance Agent (http://www.remem.org) is an application that allows for mass document storage and contextual relevance-based document retrieval. It allows a user to take any text document that they find interesting or important such as conference papers, technical datasheets, old emails, etc. and create a contextual database of those document's contents. Users can then run queries on that database with a text string from their current document and get in return recommendations for archived documents that are relevant to the text string. In short, the Remembrance Agent allows computers to do what they're good at - mass information storage and retrieval. It allows humans to do what we're good at - piecing together disparate pieces of information in some intelligent, useful way.
The Remembrance Agent was originally designed as a plug-in to the all-powerful Linux-based word processor Emacs. When enabled, the Remembrance Agent would periodically take the latest 100 or so words from the current text file and run a query with that text. Suggestions for relevant documents would be displayed in a small window at the bottom of the screen. The goal of this project was to replicate this functionality on an internet-enabled handheld computer; in this case the Pocket PC based Compaq iPAQ.
Our goal for this project was to create a client-server application that interfaced the handheld iPAQ with a Linux-based web server in order to seamlessly mimic the original functionality of the Remembrance Agent/Emacs combination. This approach was taken in part to extend the "division of labor" metaphor that made the Remembrance Agent so useful in the first place. Where the original application split the responsibilities of the human and the computer, we would add a further layer of abstraction by combining the strengths of different types of computing hardware platforms.
The iPAQ's value is its portability. It can easily be taken anywhere, and with Internet access it grants its user constant access to vast amounts of information (subject to the rather harsh restrictions of its battery). What the iPAQ possesses in portability, however, it lacks in speed and storage. A direct, self-contained port of the Remembrance Agent to the iPAQ might be possible, but it certainly wouldn't be very useful given that the user would be limited in both the number of documents they could store and the speed at which they could run queries. A desktop PC, on the other hand, allows for nearly unlimited document storage and processing power, yet is distinctly importable. To interface the two platforms, such that the iPAQ served as a graphical front-end to the PC-based Remembrance Agent installation would bring real added value to the existing Remembrance Agent project in that it would enable users to have instant, and more importantly, intelligent access to their important documents and correspondence anywhere they go.
The implementation of our project was to contain two principle features. The first feature would be the suggestion of relevant documents from within the iPAQ front-end. This was achieved with a high degree of success. The second feature was to enable users to easily add documents culled from the web to their Remembrance Agent database. This will be implemented by means of dragging the text of a document's URL to a special region of the screen to add that document to the database on the PC. Due to the limitations of the iPAQ's interface, specifically its inability to drag text, this feature was abandoned for now. Instead we setup this function to work by explicitly calling the CGI script needed to add documents to the database.
The Remembrance Agent client is a multithreaded Windows CE application. The user interface consists of a window for text editing. A thread that is constantly monitoring what is typed in the text window queries the Remembrance Agent server with the words in the text window as the query. Any returned results, along with their relevance score, are displayed in static text boxes below the text editing window. These static text boxes may be clicked which sends a notification to the parent window. The parent window then issues a call to the ShellExecuteEx API function which launches the Internet Explorer browser and opens the desired document. If one does not wish the application to issue queries to the RA server, "No Link with RA" can be selected from a combo box in the tool bar which halts the RA query thread.
The iPAQ communicates with the PC-based Remembrance Agent via a CGI-enabled Apache web server on the PC. Two CGI scripts were written to act as communication pathways between the iPAQ and the PC. The first, called ra_query_ez.cgi would take a text string as input, such as recent text from the user's document, and initiate a Remembrance Agent query using this text as the search data. The script would parse the query results and return to the iPAQ an HTML-formatted string that contained the number of suggested documents with the Internet location, filename, and search score of each of these documents.
The second CGI script, called get_doc.cgi allows a user to add documents that they've found on the web to the Remembrance Agent's database. The script takes as input a fully formed URL to the PDF/PS/text document, and then downloads that document to the local storage drive. The script then converts the document to text as the Remembrance Agent can only work with text files. The script then instructs the Remembrance Agent to re-index its database with the new document.
Several challenges were met during the implementation of this project. The principle challenge was the inability to click and drag highlighted text in the iPAQ. This is a common feature of desktop and laptop PCs, but is nonexistent on the version of the iPAQ that we used. As such, we were unable to elegantly pursue a feature that would have allowed users to easily add documents to their Remembrance Agent database from the iPAQ interface. Documents can currently be added to the PC database by explicitly calling the script from the iPAQ's web browser. The get_doc.cgi script, which performs the server-side duties of this feature, was written so that once the highlighting and dragging features are added to the iPAQ users will be able to simply drag the URL to a specified box to add that document to the database. We predict that the next generations of Pocket PCs will certainly have the features needed to use our application.
The inefficiency of the iPAQ's text input mechanism proved to be a serious usability flaw. Without the presence of an external keyboard or good handwriting recognition, there is simply no good way to input text to the iPAQ, thus rendering the need for our text editor or, for that matter, ANY text editor essentially non-existent. Again, with the development of newer generations of Pocket PCs and improved handwriting recognition software our application will most definitely become very useful.
The immense power demands that the 802.11b wireless card placed on the iPAQ were frustrating, as they both hindered our ability to test the application and called into question the feasibility of a network application on a device that has insufficient battery life to stay on a network. We are confident that with time power requirements of wireless adapters will drop and battery life will improve, they will have to make the iPAQ a useful tool to explore the Internet for a longer period of time. Once this is achieved our project will also have the required power to be used effectively.
Despite the hardware limitations of the early-generation iPAQ, we believe that our project was successful. As the power, storage and speed of handheld computers continue to improve, the client-server model that we've developed for using the Remembrance Agent should remain relevant given the reliability and physical security of a desktop PC as compared to that of a handheld.
With further development, our project could be extended to integrate with existing word processors, such as Microsoft Word, which would allow for its use by more individuals. It could also be modified to provide optional additional information about the suggested documents, which would help to "jog" the user's memory and better determine the relevancy of the suggested document without having to actually open it.
The PC-side of this project requires several external applications. All applications listed here are free and open source to the best of our knowledge.
Also be aware that you will have to modify the two CGI scripts (ra_query_ez.cgi and get_doc.cgi) to reflect the particulars of your system, such as your Internet hostname, directories for permanent document storage and the Remembrance Agent directories. The variables that you will need to change are located near the top of the scripts in a section labeled "System Variables."
The two CGI scripts (ra_query_ez.cgi and get_doc.cgi) should be placed in the CGI directory of the user's public web directory (usually