This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015.
I can be contacted at email@example.com.
History of Pen and Gesture Computing:
Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers
References from the approximate years 2014 to 2015.
Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement:
Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward
Most of these are in my personal collection, either as physical items or electronic files.
Copyright © Fri Apr 24 17:23:54 EDT 2015.
Mirrors of this page can be found at:
biblio13 (Prev page)
(Next page) biblioxx
Reflective infra-red IR photodiode illumination, optical sensors recognizing hand gestures in front of display screen. Up/down/left/right north/south/east/west gestures, activated by proximity detection. Also detect proximity of smartphone to user's ear. Includes broadcast of 1D barcodes using LED (compare with Morse code).
U.S. Dept. Edu. historical list of screen readers and voice output systems for visually impaired: 26 pages. Historial list, many products discontinued before 2002: AppleWorks Companion, DragonDictate, Vocal-Eyes, etc.
Touchscreen/touchsurface controllers for touch panel overlays/underlays for "any material" (wood, glass, metal, fabric). Piezo single-layer disk (stacked?), for both sensing and haptic feedback actuator, including audio. No interpolation for position(?)
Double/multiple transparent display screens, one is touchscreen: basically a transparent tablet computer. Show a transparent window in the screen, overlay graphics on real-world image for augmented reality applications. Video overlay possible.
Biometric authentication of users: send known acoustic signal through body, subtract original signal from received signal to get modification, identify user by unique modification/alternation to signal. e.g. at a doorknob to grant entry. Signal can also pass through two users (holding hands) to transfer data. (Acoustic, not capacitive coupling like at M.I.T., Rekimoto, etc.)
NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Lower electrical resistance than ITO, more transparent (0.6%); less reflected light than silver nanowires.
NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Low electrical resistance.
Circular optical multi-touch touchscreen: optical sensors around entire circumference, smaller number of (infra-red) light sources around, checks for occlusion of light to sensors. No reference to shadowing problem.
Live video (from on-board camera of actual environment) on a tablet computer with touchscreen input: annotations / electronic ink information layer overlaid on live video as augmented reality.
Software framework package for tactile output desplays (for the blind), so that custom software does not have to be written and developed for every devices. Internally implements multiple screens (windows?), user can switch applications quickly. Keyboard and arrow keys, or touch tablet surface (cites to others). Basic gesture recognizer for pointing, swipes, pinch, circles, multi-finger/multi-touch drag. Display is a viewport onto a portion of the nominal visual display.
Tactile output desplays (for the blind) and for the sighted for collaborative work: extension of tabletop/tablet to visually impaired. Pin-matrix display (BrailleDis7200), touch input. Button and arrow-key rocker user interface for blind (so that touching the display to see the display does not cause user input?).
Essay on design process for a tablet-based reader for reference material: not page-oriented (as PDF), remembers recently accessed information. Two-dimensional scroll: vertical for coarse/regular positioning, then horizontal motion for fine possitioning for long texts. Suggest design layout and features first on (larger) tablet, later user interface adapt for smaller smartphone/screens: once you know what it important.
Essay piece: Three trends in machine control interfaces: multi-touch touchscreen sensors, communication with mobile devices (or more correctly, from mobile devices e.g. smartphones and tablets to industrial equipment in a factory), and better human factors in the software.
Silver nanowire coating material, alternative to ITO indium tin oxide. File contains additional materials: Technology Review articles, etc.
Experimental system for combining touch/multi-touch gestures with in-air proximity gestures, e.g. high-jump gesture of two taps with finger going high between to select; in-air pigtail hand/finger gesture to copy to clipboard. Prototype uses depth camera over smartphone touchscreen: capacitive/electrostatic touchscreen sensing to proximity from 1.5 to 4.0 cm.
Touch-to-search gesture: bi-manual / two-hand gestures used for search user interface: user holds touchscreen with touch on one side, other finger can highlight/select text for search on web. Example: eaiser to find other web pages about a product listed on Amazon.com. (Microsoft Surface)
Reverse-offset printed mesh/grid of silver/Ag nanoparticle conductive ink trace wires for touchscreen touch panel: single layer printing on flexible plastic substrate. At 10-micrometer width, invisible (transparent) to human eye. Capacitive/electrostatic sensing, does not give details of single-layer sensing -- cites to Ho 2009? Illustrations show grid with 10-micrometer breaks in traces, no sensing pad matrix array?
Optical multitouch touch tablet, thin profile. Light emiters inject light from points around the sides, detected by sensor on other sides. Touch causes weaking of light along path due to FTIR total internal reflections. Not clear how it deals with shadowing problem for multiple touches: multiple sensing paths in different directions. Compare with Rapt Touch?
High-reliability measures for capactitive grid (multi-touch) touchscreens for avions, other safety-critical application. Basically detect a broken trace or line in the grid, and use the average/combined sensing of the two ajacent conductors.
Transparent touchscreen using grid of polymer waveguide/lightguides figers with monochromatic light: finger touch affects FTIR total internal reflection, light along that fiber is "shadowed".
multi-touch optical touchscreen, infrared light beams to multiple receivers at edges of glass, touch affects internal reflection within glass. Mentions palm rejection, no details. See Christiansson patent.
3D three dimensional manipulation using a flat 2D multi-touch surface: 3D images are shown below surface, with a simulated vertical "shadow" (cone shaped projection, narrowing as comes up) from virtual object up to touch surface. User can manipulate object by multi-touch on shadow at tabletop. Eliminates visual convergence problems, since fingers and shadow "surface" are on same plane.
Essay on "cooking" of experimental statistics (examples mostly in the social sciences) by phrasing questions after the collection of data: questions can be chosen/re-phrased for any particular data collection in such a way to appear to support almost any hypothesis,
Tactile/haptic feedback on electrostatic multi-touch screen by driving X and Y lines for a particular point for feedback at frequencies above human detection (1000 Hz), beat frequency is perceptible (240Hz). Not highly localized, perhaps usable for keyboard images. Appears to be able to do only one location -- shadow effect?
Video image of hands combined semi-transparentlly with computer image, so that user perceives hands as touching directly on image, without hands obscuring image. Any touchpad/tablet technology: also proximity sensing? Compare to Wellner, systems with rear-surface touching, 3D manipulation / virtual reality systems.
Combine multi-touch hand and stylus (e.g. able to sense orientation, type of grip) for gestural context based on behavioral context. Ignore unintentional touches (palm rejection) by context. Two-handed multi-model gestures: e.g. pinch gesture with while tucking pen in hand gives magnifying glass for drawing; touch/point with left hand while angling stylus above surface gives airbrush tool. Stylus grips: writing, tuck (stylus held like chopstick while writing hand is gesturing), palm (stylus held in fist), extension (point with a finger while tick or palm grip). Review of user-interface work on grip sensing, proximity, 3D position or orientation of stylus, hand gestures. Handing tablet to other user brings up per-user acetate drawing layer.
Reference text on computer graphics: similar to Foley (also contributor). Described alpha blending as "coverage" (opacity/translucency/transparency).
Marketing and product information on touchscreen display with capacitive multi-touch integrated: appears to be sheet of ITO within glass on front of display elements/LCD, matrix of scan electrodes within glass on rear.
Bookmark pages temporarily in electronic book by touching them with multiple fingers, like gesture of holding finger between pages. Multi-touch touchscreen.
Commercial (Elo Touch Solutions) appeal that capacitive (projected) touchscreen technology will dominate, but not appropriate for all solutions. P-cap difficulty with any stylus (vs. resistive, optical), contaminant (e.g. water) rejection, high transparency (vs. SAW and optical), extremely large surfaces (resistive, SAW).
In-air gestures using both position and orientation of a hand-held device (WII?), for drawings/markings (graffiti) on image.
Electrostatic grid touchscreen, reduce power consumption by scanning fast/coarse until a touch is detected, then scan at full resolution. Compare with Pencept/Numonics course/fine scanning on electromagnetic tablets.
Proposal for combined fingerprint/multi-touch touchpad device: high resolution matrix, scanned at low resolution (every "N" trace lines) for touch input: for fingerprint input, when finger is found, re-scan just that area at high resolution (don't skip any traces). Can be implemented in-cell (with display) by sizing electrodes/sensor points as sub-pixel spacing.
Electrostatic sheet touchscreen, single-touch: User finger is active probe, signal input by wrist-strap: ratiometric measurement of signal picked up by resistive sheet. Appears very similar to Scriptel.
Flexible touchscreen display sensor, matrix of small printed strain gauges. Compare with flexible/deformable input devices.
Transparent touchscreen with display, two users collaborating on opposite side, can switch touchscreen among transparent (direct communication), translucent/semi-transparent (collaboration and video), and opaque (displays for both users only). Not sure it comments on mirror-image/reversal between users. Compare with Whiteboard/collaboration systems.
Capactivie linear displacement and capacitive position sensors, primarily for industrial use: mostly linear sensors, not X/Y. Also eddy-current/inductive linear sensors. Includes TechNote L03-0020, Capacitive Sensor Operation and Optimization tutorial.
Single-layer capacitive touchscreen (not multi-touch) with one layer of electrode strips: X coordinate by which trace finger is near, Y coordinate by modeling resistance along trace as two resistors divided by a capacitor to ground where finger is along the trace.
Shake or tilt a smartphone (shake/tilt gesture), as an indication to detect user mistakes on user interface, and adjust the user interface (move/shift/re-size icons).
Slides of continuing workshop series NIME ("New Interfaces for Musical Expression", since 2001 CHI) on musical instrument interfaces. Historial discussion of theremin with capacitive proximity sensing. Haptic, touchscreen, force-feedback, force (pressure) sensing, torque/rotation/translation forces (side-ways force), proximity, Polhemus and Ascension flock-of-birds 3D sensors. TouchGlove and TouchStrip, TapTile FSR (force-sensitive resistor). Visual-input piano (optical/video sensor), ReacTable movable puck/object sensors. Kinect/ Hugh Le Caine keyboard touch sensor 1940s.
Two Video presentations: one for conference, other as Webinar. "Turn any surface into touchpad": using Google glass or other optical head-mounted sensor, detect residual heat thermal signature on surface briefly after touched by human finger. Requires that optical camera respond to / distinguish infrared?
Prototype / proof of concept only, company (German) does Google glass / virtual reality applications.
File contains additional materials.
(in German) HyperBraille/BrailleDis 7200/9000 product information and technical sheet. 60x120 pin-matrix graphical display for the blind, includes touchscreen input with 1440 touch sensors. Also Modul D2 data sheet: 2x10 single-character Braille output, building block for BrailleDis with 2 touch sensors each (capacitive?).
Fine Arts faculty profile for Slavko Milekic. Artistic presentations use gaze-tracking, touchscreen input. Mentions KiddyFace for art exploration with children.
Single point (one fingertip) haptic/tactile in-air feedback using focused ultrasound, combined with half-mirror 3D three dimensional display (Aerial Imaging Plate API ASUKANET commercial unit). 40 kHz signal has focal spot about size of a fingertip (one wavelength), modulated at 100 Hz to produce sensation. Compare with UltraHapics project at U. Bristol.
Tablet/hand-held computer (smartphone) with second touch screen on the back/rear side, with rotation/orientation sensor. If held vertically, only bottom half of rear touchscreen is active.
Sides of touchscreen mounted on small pillars, can detect horizontal (and vertical) shear forces. User interface to deform virtual objects. Compare to Margaret Minsky?
Tuturial/Overview on LabView system and graphical programming language, 2014. Focuses on programing of virtual lab instruments in LabView VirtualBench. Describes VirtualBench APIs: the graphical programming interfaces between LabView virtual instrument (VI) objects.
Ocular: Projected capacitive / electrostatic touch panel tablets: Crystal Touch / Edge. Thin glass. Optically-bonded glass cover optional intead of air gap between layers. Shows flush and bezel mounting. Includes multi-touch demonstration software with finger tracking. Recommends touch areas (accuracy?) of 10 mm plus spacing. Mentions two-finger pinch/zoom without multitouch.
Optical/camera detection of hand gestures in a smartphone, for controlling a projector built into the phone (or other device). Waving hand left/right to scroll. Basically only detecting gross motion of any kind, not discrete gestures.
Penclic Mouse B3: Mechanical roller mouse in form of pen with roller in tip, compare with PenMouse from 1980's (in collection). Wireless bluetooth interface. www.penclic.se product literature in file includes cable versions. Multiple barrel buttons for left/right mouse button, center roller, page up/down. Illustrations seem to imply detection of rotation of stylus/pen.
Penclic external touchpad: with four buttons for left/right click, forward/back keys (for browser), additional physical spin wheel /scroll wheel for scrolling.
User-interface combining eye-tracking (gaze) with multi-touch gestures on touchscreen: Gaze/eye-tracking selects, touch manipulates. Gestures can be made at any location on objects selected by gaze "dwell". compare with Wellner etc. large wall displays. Problems noted include multiple-select (essentially lose multi-touch for select), "muscle memory" gestures (activating PIN keypad) etc. where user is not gazing at the object to be manipulated.
Optical sensing touchscreen: optical emitters at sides direct beams withing glass to multiple optical sensors spaced on opposite side, touch on surface of glass disturbs (reduces) frustrates total internal reflection (FTID). Comparision of disruption of multiple beams with pre-analyized disruptions by known reference touchs, determine multi-touch inputs. See Rapt Touch file (assignee). Compare with FlatFrog?
Resistive/consductive films, 200-micrometer-sized metal mesh in polyester/PET film substrate, alternative to ITO indium tin oxide, moldable to curved surfaces. "Touchscreens mit transparenten, flexiblen und leitungsfäigen Folien"
User study of BrailleDis 7200 tactile pin-matrix display for the blind. User interface has multiple controls: on-screen sliders and buttons, function keys, cursor keys, separate touchpad. Function button to switch to gesture input (vs.s. "view" mode where user scanns display). Separate touchpad not as good as touchscreen input.
"Touchless Gesture control": Passive infra-red optical video sensor for detecting hand and body gestures: comntrasted with touchscreen and with Microsoft WII.
Optical touchscreen using radial pattern of internally-reflected light beams through glass layer. See Piot reference for details. Also see video file, same web site.
Very small touchpad keyboard for watch-sized devices: long-tapping vs. short-tapping: holding finger on keyboard for longer time with zooming UI display allows for more precise input of characters, system then knows not to do word-substituion/spelling correction.
ITO-replacement, streatched carbon monofilament film on polycarbonate substrate. Largr-format tabletop/touchscreen, multi-touch (up to 10 touches) projected capacitance.
3-D display (of sorts): flat display suspended from movable support arm on ceiling, user can move display in 5 DOF degrees of freedom (X,Y,Z,pitch,yaw -- no roll) to see views of volumetric images. More importantly, display can move autonomously without user's hands: e.g. user requests view of something (e.g. a particular portion by name of a 3D image of a brain), display moves autonomously to position and orientation to display view. Uses in-air gestures (e.g. in-air pinch gesture to "pick up" display so that it is moved in direction of hand motion.
Long-term project review (back to 2001) for thin flexible electronic skin E-skin touch sensor, proposed uses are robotic sensing, medical instrumentation. Sensor skin applied to throat to recognize speech by muscular activity. Plastic substrate, TFT touch sensors. Shows pressure profile from array of sensors.
Tactile/haptic electrosensory display. AC-modulated signal coupled capacitively to fingertips at discreet pads, sensation is direct Coulomb force capacitive to Pacinian mechanoreceptor nerve cells in fingertip, not physical vibration or electrostatic friction.
Matrix pressure-sensing tablet, piezoresistive grid, minimum sensing grid resolutation approx 1.6 mm. Matrix can be flexible, and also stretchable (elastomeric). Targeted as tactile sensor for mechanical testing, measurement of fit (of chairs, mattresses, etc.).
in-air/proximity gestures on smartphone using regular smartphone camera on front. States advantages for mobile use, hands-free, display screen not obscured. Continuous recognition of pinch/zoom gesture motions of hand (move in and out, etc.). Compare hand/gesture detection with Wellner? Mentions touch+gestures (see files).
micro-fluidics channles in thin layer, can be inflated to create physical buttons on transparent touchscreen.
3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick?) for pan, zoom rotate.
3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick?) for pan, zoom rotate. Five near-mouse function keys, use short press (tap) and long press (press-and-hold) for 10 functions. Function keys automatically redefine function according to appliction (default function of OS?). Keyboard modifier keys (shift,alt,ctl) that are often used with mouse input. as qualifiers/modifiers.
3D three-dimensional "mouse" input device Sales brochure (?) on CAD services and software: Third-party Product Data Management PDM tools for collaborative 3D CAD design work: GrabCAD Workbench, Tinkercad, Sketchfab, WikiSpaces, HackPad, etc.
Data Glove for industrial applications with gyroscopic sensor, accelerometer, contact detection sensor on index finger: compare with Micorosoft WII controller? Motion sensors used for gestures, contact sensor and tag reader (RFID?) used to recognize labels on equipement. Intended as wearable data entry device.
transparent capacitive-grid (multi-touch) touchpad film, overlaid on keyboard. "Machine learning" to distinguish moving touchpad input and keyboard typing input. Call capacitive sensing grid a capacitance-to-digital converter. Cites to Rekimoto ThumbSense, but not to multitouch tabletop.
Wall-sized display with tactile feedback (vibration)
Transmit additional information through existing capacitive/electrostatic touchscreen: circuit in signet ring on user's hand generates signal that can be picked up by existing touchscreen electronics at low data rate (5..10 bps). Can also be used to identify multiple styli/styluses, multiple users, etc.
Review of produced/exhibitors at Display Week exposition: embedded-touch and in-cell touchscreens and sensors, integrated with display. Reverse-side electrostatic/P-CAP; single-layer mutual-capacitance using "caterpillar pattern"two-element grids with crossovers/bridges; "Pixel Eyes" (JDI vendor); optical fingerprint sensor, very high resolution, LED pixels used for touch sensing. Water resistance: does not affect self-capacitance sensors, only mutual-capacitance. Electromagnetic resonance touch battery-less unpowered stylus from Hanvon: compare with Walcom? ITO replacements carbon nanotubes, silver nanowires, silicon-doped zinc, PET PoliyIC film. photolithography printing of mesh. Resistive-film multi-touch using 12 mm sensor pads.
Encryption/DRM application platform (with API) for protecting multimedia (especially Video) documents and content. Uses secure hardware for keyboxes to store private PKI keys.
2D and 3D vector graphics library, includes alpha blending. API specification for access to hardware.
BitBlt binary block transfer operator/operation.
halftone image typically 300 dpi or 65 lpi resolution on laser printer, lower on display
Standard structure of an application (look-and-feel / GUI). OS-specific examples (MacApp), Cocoa (iPad), also cross-platform (Qt, wxWidgets, etc.), NetBeans, KDE.
General information on Multics OS "Multiplexed Information and Computing Services": access control lists on files/objects, all files memory-mapped.
Generalized concept for application framework: includes inversion of control, default behavior, extensibility, non-modifiable framework code
Study of resolution/accuracy of haptic feedback using ultrasonic acoustic radiation pressure. Discussion of various nerve sensors: Pacinian corpuscles (vibration); mechanoreceptors have higher acuity. Hand-gesture sensors and tactile feedback mounted on user's hand.
Experimental system using 3D "hover" information on fingertip to predict point of contact, and signal touchdown with less latency. Microsoft Surface (Samsung SUR40 tabletop / PixelSense) with 3D camera tracking system of marked fingertips.
Haptic feedback (simulated traction/friction/repulsive forces) on stylus, using asymmetric vibrators for any X/Y direction. Cites to Traxion by Rekimoto. Suggests using force to guide/teach user, rather than to simulate feel of virtual objects (brush, etc.), due to limited angular accuracy of prototype.
Real-time and asyncronous (recorded) document review system including electronic ink annotations, syncronized voice, pointing, gestures. Hovering places "highlighter" color for pointing. Cites to Wang Freestyle: Freestyle could also do pointing gestures via "hover". Users spontaneously allocated particular space for comments: bottom of page for general comments on page, to side of text for comments on particular text. Compare with Anoto SmartPen? Space can be expanded for additional ink (+ voice) annotations: compare with van Raamsdonck?
Interesting study of abilities of service dogs to learn to use a touchscreen interface: applications include calling emergency numbers, etc. Also found that ability to learn and perform this task ("drive") predictive of ability to focus and work as guide guide dog (to save training expense on dogs without enough "drive"). Authors appeared confused about multi-touch vs. single-touch touchscreens: switched to IR/LED optical touch screen (not identified) because saliva and moisture from dog's snout (one dog spontaneously used paw) affected capacitive touchscreen.
Proximity sensor used for hand gestures on smartphone: additional antennas pick up reflect GSM signal from smartphone itself, hand proximity affects signals received by multipath reflection/propagation. Recognize in-air swipe gestures, proximity on different edges and in front.
biblio13 (Prev page)
(Next page) biblioxx
This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015.
I can be contacted at firstname.lastname@example.org.