Henry Rzepa's Blog

Augmented reality, a superset if you like of virtual reality (VR), has really been hitting the headlines recently. Like 3D TV, its been a long time coming! Since ~1994 or earlier, there have been explorations of how molecular models can be transferred from actual reality to virtual reality using conventional computers (as opposed to highly specialised ones). It was around then that a combination of software (Rasmol) and hardware (Silicon Graphics, and then soon after standard personal computers with standard graphics cards) became capable of such manipulations. VRML (virtual reality modelling language) also proved something of a false start^‡ So have things changed?

Many of the posts on this blog have some element of such VR in the form of the Jmol or JSmol software (the natural successor to Rasmol) that allows a 2D projection of a 3D model to be manipulated in "real-time", allowing the geometrical features to be inspected and even animations of reactions. Google cardboard is a (minor?) variation on the VR theme, allowing a 3D object to be viewed through a simple cardboard headset containing a mounted phone, but controlled by head movements acting on the accelerometers in the phone rather than a mouse or trackpad. But the full-blown experience is something else, and watching this TED video really brought it home to me. The virtual object, such as say a molecule, is superimposed upon one's view of the real world (AR) and this object can now be controlled with hands as well as eyes. Again, this is not new; so-called haptic control of virtual objects has been around for a decade or more, in which you can e.g. probe how "hard" an object is using a haptic or hands-on device such as a joystick. All of this quickly convinces one that the secret of successful use of VR and now AR to augment chemistry is going to be the software!

We now need inspired programmers to create the Rasmol/Jmol of augmented reality. But beyond mere software, chemistry with AR needs to be placed into the appropriate environment or context. One might presume this will include the stereoscopic video inputs from other AR headsets (the research team, the collaborators, etc) but what else? The pages of a blog? Or a journal article? Could indeed one recast the journal article itself into an AR scene, with the various components floating in space, with molecules conjured out of a table (or a synthetic procedure) to float in full 3D glory to be played with by the participants? I rather suspect this might be quite a few steps too many for most!  Think how little ~22 years of the Web (and perhaps ~36 years of the Internet itself) has actually changed the construction (I do not mean the delivery) of the average scientific article. Even now, tables in which molecules can be treated interactively are extremely rare. Most of this is because authoring tools such as Microsoft Word have not yet made the production of such documents viable. So perhaps the augmented-reality scientific or chemical article may not be quite around the corner. Perhaps the AR hype will end in the same way that 3D TV appears to have. But unless we experiment, we will never know the answer. So if any reader of this blog knows of interesting work in chemistry AR, do drop me a line.

^‡ Virtual Reality Modelling Language (VRML) in Chemistry, O. Casher, C. Leach, C. S. Page and H. S. Rzepa, Chem. in Brit., 1998, 34(9), 26. But VRML has made a come-back as the language of choice for 3D printing!