This portion of our Spatial Reasoning with Robots project is to implement an interface within which we can intuitively sketch an environment in which the robot will interact. The sketch will provide the robot with a mapping of drawn objects, and a path with which to navigate those objects.
The challenge of the sketch interface will be to intelligently, and consistently, interpret the markings of the user. The goal is to have the user require as little knowledge of the interface as possible. The user should be able to draw and/or mark out objects and paths in such a way that the use of menus or commands is unnecessary. The sketch interface should be able to interpret the markings of the user and transform the sketched image into a map environment within which the robot should navigate.
With work done thus far, a palm pilot based sketch interface that can input a user drawn sketched map has been previously reported. There are some limitations with this early system though. One deficiency requires delimiters to mark each new object. The user must explicitly input these between drawing the objects. The user also had to denote the path in the original implementation. This approach is not an intuitive process for the user. We are now redesigning the system such that the sketch interface determines the different objects and the pathway based upon the drawn sketched markings.
A second limitation was if the user wished to remove an object from the drawn sketch. There should be no need to start over with a clean slate. It would be more convenient for the user to have the interface interpret a crossed out object and remove it from the sketch, obviating the need to redraw the entire sketch. Other markings, such as indicating the starting point of the pathway, might be useful to interpret as well. These operations are currently being added to the sketch interface.
The ability to label the objects in the drawn map allows for further interaction between the user and the robot in the context of its environment. After an object is labeled, the robot can now refer to that object by name. This allows us a method to interact with any object recognition systems that the robot may possess. Indeed, we are proposing to incorporate such behaviors into our robot system with the morphological shared weight neural networks (MSNN) for image recognition.
Another addition which we are implementing is to have the palm based sketch system interact wirelessly with the spatial reasoning system to input the drawn sketched map seamlessly into an existing TCP-IP based client server system. This will allow real-time sketches of the environment to be input to our robot from a distance.
Of course, we are not limited to palm pilot devices for sketch input. A new generation of tablet based PC's are now available. The additional screen area and more powerful processors on board these machines will allow for even more innovation in the utility of sketched diagrams. In fact, a sketched diagram could be input from literally any computer with an appropriate drawing program and a network connection. However, the pen based approaches of the palm and tablet computing platforms provide for a very intuitive input system for a user in the field.
This sketch-based system leads us to many exciting, and important, applications for such a system. For example, search and rescue robots in which a remote user might have some data (i.e., a camera image or sonar reading) with which to draw a sketch of the surroundings. The sketched interface could potentially provide a more robust description of the surroundings as it reduces the amount of unimportant scene descriptions. This helps to identify the more important landmark objects that much of navigation depends on.