This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at email@example.com.
History of Pen and Gesture Computing:
Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers
References from the approximate years 2014 to 2015.
Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement:
Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward
Most of these are in my personal collection, either as physical items or electronic files.
Copyright © Tue May 24 19:30:52 EDT 2016.
Mirrors of this page can be found at:
biblio13 (Prev page)
(Next page) biblioxx
Reflective infra-red IR photodiode illumination, optical sensors recognizing hand gestures in front of display screen. Up/down/left/right north/south/east/west gestures, activated by proximity detection. Also detect proximity of smartphone to user's ear. Includes broadcast of 1D barcodes using LED (compare with Morse code).
Finger stylus (sock-style conductive stylus over a finger), conductive head pointers, mouth sticks as accessibility devices for use with capacitive touchscreens such as iPad.
U.S. Dept. Edu. historical list of screen readers and voice output systems for visually impaired: 26 pages. Historical list, many products discontinued before 2002: AppleWorks Companion, DragonDictate, Vocal-Eyes, etc.
Fine-point conductive stylus/pens for iPad and iPhone touchscreens. Rounded tip, pressure-sensitive (tip force). Other models have "precision disk" on tip to contact surface. Bluetooth connectivity, works with any app. Notetaking applications.. Force-sensing stylus requires "compatible apps" for pressure sensitivity and palm rejection on iPad/iOS.
Reduce noise on capacitive-grid stylus/tablet by measuring noise alone on adjacent grid conductors and subtracting from stylus signal. i.e. differential sensing. (Compare with Blesser/Pencept?)
Touchscreen/touchsurface controllers for touch panel overlays/underlays for "any material" (wood, glass, metal, fabric). Piezo single-layer disk (stacked?), for both sensing and haptic feedback actuator, including audio. No interpolation for position(?)
Android touchscreen smart phone. Set-up includes creating/entering Google account for HanOuts, Google services.
Double/multiple transparent display screens, one is touchscreen: basically a transparent tablet computer. Show a transparent window in the screen, overlay graphics on real-world image for augmented reality applications. Video overlay possible.
Survey article on user authentication by keystroke dynamics (typing): earliest citation is 1980.
Biometric authentication of users: send known acoustic signal through body, subtract original signal from received signal to get modification, identify user by unique modification/alternation to signal. e.g. at a doorknob to grant entry. Signal can also pass through two users (holding hands) to transfer data. (Acoustic, not capacitive coupling like at M.I.T., Rekimoto, etc.)
NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Lower electrical resistance than ITO, more transparent (0.6%); less reflected light than silver nanowires.
Anoto pattern smart-pen Live Pen application: Live PDF / Live Paper app prints PDF forms with anoto pattern, smart pen sensor can then capture "wet signatures" and handwriting. Works with @LiveAddress storage of document definitions.
Gestures on OSX trackpad/touchpad digitizer: two-finger zoom, single and multi-finger swipe and tap gestures. Control panel checkboxes to enable/disable features.
user-interface UI guidelines for iOS 7 touchscreen operating system. Dynamic text to adjust letter spacing and line hight automatically. Automatic layout for size/zoom left to application: suggest avoiding fixed positioning. Icon/target sizes given in pixels 44/80 minimum. (at 240 pixels/inch?) Double-tap to select a word in text. Shake (smartphone) to undo gesture.
Mouth-stick accessibility stylus for capacitive touchscreens. Offered on Apple web site, third-party product.
User guide for VoiceOver: Chapter 11 Using VoiceOver Gestures: area of screen affected is highlighted to make it easier for sighted to assist. Commands assignable to gestures; rotor -- two-finger press-and-hold brings up rotor for selection navigation options: Web, Word, etc. VO/shift key used to modify gestures. Flicks, taps, rotate with 1, 2, 3 fingers.
Apple gestures for Mac multi-touch trackpad digitizer: pinch-to-zoom, single and multiple finger tap and swipe gestures.
Motion (for OS/X and MacIntosh) video animation/editing software. Large number of shape-based gestures: i.e. circle left and right for zoom in/out. Compare with Nokia gestures, Squeak mouse gestures? See also Motion 2.
Apple Pencil stylus: announced with iPad Pro. Stylus for use with iPad Pro touchscreen: tip-force (pressure), stylus angle (2D, not rotation?). Stylus sensing is optical(?), compare with Anoto? Includes marketing demonstration video.
API call on apple iOS for getting diameter/size of touches on multi-touch touchscreen. Used in some third-party products for tip-force/pressure, but tip-force/pressure is an additional value. Extensions fo stylus angle for Apple iPencil?
NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Low electrical resistance.
capacitive touchscreen, both self-capacitance and mutual capacitance modes, adapt for water droplets on surface, gloves, proximity sensing. Compare with Apple/Fingertouch?
Evaluation board multi-touch capacitive touchscreen, both self-capacitance and mutual capacitance modes. Includes diagram of peak detection.
Single-layer capacitive touchscreen with multi-touch for fingers, separate detection of stylus. Stylus reports pressure/force on tip. File also contains info on QTouch low-power controllers, operate automatically in mutual-capacitance/self-capacitance modes to deal with water droplets, gloved hands. Design information on hardware embedding (button size vs. icon, etc.)
Touchpad used for full screen by letting user throw or glide cursor, motion stops when user touches touchpad again. Cites to kinetic scrolling, momentum in GlidePoint touchpad. Compare with virtual physics, artificial reality project. Cites to screen-edge motion.
Circular optical multi-touch touchscreen: optical sensors around entire circumference, smaller number of (infra-red) light sources around, checks for occlusion of light to sensors. No reference to shadowing problem.
Live video (from on-board camera of actual environment) on a tablet computer with touchscreen input: annotations / electronic ink information layer overlaid on live video as augmented reality.
Haptic feedback depends on gesture made. Feedback may be applied to multiple interconnected devices: e.g. to a remote whiteboard display/system or drawing surface. Configuration for haptic effects stored in a file.
Overview of touchscreen/tablet technologies, comparison charts. Includes in-cell resistive multitouch tablets, reverse-diffused optical, FTIR, etc. No mention of electromagnetic, acoustic/sonic, only tablet technologies author said would work with an (ungloved) finger touch. Compare with Geoff Walker?
Software framework package for tactile output displays (for the blind), so that custom software does not have to be written and developed for every devices. Internally implements multiple screens (windows?), user can switch applications quickly. Keyboard and arrow keys, or touch tablet surface (cites to others). Basic gesture recognizer for pointing, swipes, pinch, circles, multi-finger/multi-touch drag. Display is a viewport onto a portion of the nominal visual display.
Tactile output displays (for the blind) and for the sighted for collaborative work: extension of tabletop/tablet to visually impaired. Pin-matrix display (BrailleDis7200), touch input. Button and arrow-key rocker user interface for blind (so that touching the display to see the display does not cause user input?).
Inking stylus for capacitive touch screens: see 9,092,077
Inking stylus for capacitive touch screens: has retractable inking tip as well as conductive rubber tip, or angular-shaped hard rubber conductive tip to put flat surface on touchscreen.
Essay on design process for a tablet-based reader for reference material: not page-oriented (as PDF), remembers recently accessed information. Two-dimensional scroll: vertical for coarse/regular positioning, then horizontal motion for fine positioning for long texts. Suggest design layout and features first on (larger) tablet, later user interface adapt for smaller smartphone/screens: once you know what it important.
Essay piece: Three trends in machine control interfaces: multi-touch touchscreen sensors, communication with mobile devices (or more correctly, from mobile devices e.g. smartphones and tablets to industrial equipment in a factory), and better human factors in the software.
Silver nanowire coating material, alternative to ITO indium tin oxide. File contains additional materials: Technology Review articles, etc.
Recognize in-air gestures using regular microphones and speakers in two smartphones, using doppler shifts on both devices. In-air swipe/approach gestures in different directions (including center-out and in-to-center). "Waving" files from one device to another. Cites to other doppler/acoustic in-air hand gestures.
Experimental system for combining touch/multi-touch gestures with in-air proximity gestures, e.g. high-jump gesture of two taps with finger going high between to select; in-air pigtail hand/finger gesture to copy to clipboard. Prototype uses depth camera over smartphone touchscreen: capacitive/electrostatic touchscreen sensing to proximity from 1.5 to 4.0 cm.
Touch-to-search gesture: bi-manual / two-hand gestures used for search user interface: user holds touchscreen with touch on one side, other finger can highlight/select text for search on web. Example: easier to find other web pages about a product listed on Amazon.com. (Microsoft Surface)
Reverse-offset printed mesh/grid of silver/Ag nanoparticle conductive ink trace wires for touchscreen touch panel: single layer printing on flexible plastic substrate. At 10-micrometer width, invisible (transparent) to human eye. Capacitive/electrostatic sensing, does not give details of single-layer sensing -- cites to Ho 2009? Illustrations show grid with 10-micrometer breaks in traces, no sensing pad matrix array?
Optical multitouch touch tablet, thin profile. Light emitters inject light from points around the sides, detected by sensor on other sides. Touch causes weakening of light along path due to FTIR total internal reflections. Not clear how it deals with shadowing problem for multiple touches: multiple sensing paths in different directions. Compare with Rapt Touch?
Modifications to mouth stick to operate as (conductive) stylus for capacitive touchscreen tablets and smart phones.
High-reliability measures for capacitive grid (multi-touch) touchscreens for avionics/aircraft, other safety-critical applications. Basically detect a broken trace or line in the grid, and use the average/combined sensing of the two adjacent conductors.
Touchscreen button targets on smartphone must be larger when user is walking around, at least 14x14 mm.
Transparent touchscreen using grid of polymer waveguide/lightguides fibers with monochromatic light: finger touch affects FTIR total internal reflection, light along that fiber is "shadowed".
Haptic/tactile output depending on touch input on a touchscreen, based on time of touch and pressure increase. (press-and-hold).
Review of tactile sensing flexible "skin" for robotics sensing. Resistive touchscreens inexpensive and simple, but not ITO not flexible, optical quality not perfect. Capacitive multi-touch high processing load. Infrared/optical good transparency, problems with environmental contaminants. Suggests flexible graphene: nothing on wire mesh. iCub capacitive using flexible (elastomeric) material between electrodes (conductive cloth). Mentions distributed-computing requirements for robotic skin.
Overview of touchscreen technologies: 5-wire touch more accurate / less drift than 4-wire touch (resistive); projected capacitive; DST dispersive signal technology senses acoustic waves caused by touch; AMR analogue matrix real (Resistive) touchscreen, divided into small separate resistive sensing zones.
Continuation of Jota 2013 paper. Effects of latency on users of multi-touch/single-touch touchscreens. Cites to lag/latency in VR virtual reality displays. Tapping latency less noticeable than dragging/direct-manipulation on touchscreens.
Hold smartphone like a fat pen (or like a stylus pointer) to write with the corner / on the wall, detect writing motion from 3-axis accelerometer readings from smartphone, thus allowing larger writing than touchscreen. Compare with H. Crane accelerometer pen (not cited).
Rockstar (see patent file), patent "troll" owned by Apple Microsoft Sony Ericsson and Balackberry drop all patent litigation against Google/Android. Includes part of patent arsenals by Samsung,Nokia, et al for 2G/3G/4G wireless standards.
Large stylus (looks like a carpenter's pencil) for iPad iOS 8. Area of contact on touchscreen, tip is rounded/pyramidal so angling stylus produced wider lines. Notes that iOS 8 has will support "variable touch sizing". Back end of stylus has large flat capacitive area, used for simulating drawing with chalk. This version passive(?), earlier 2013 version active, coupled to iPad via bluetooth. Compare with Apple Pencil, 2015.
Combined force and capacitive (or other) touchscreen digitizer, multi-touch. Two pads of mutual-capacitance touchscreen separated by compliant/elastomeric material with strain gauge sensors. Deflection of top layer greater than lower layer sensed force.
multi-touch optical touchscreen, infrared light beams to multiple receivers at edges of glass, touch affects internal reflection within glass. Mentions palm rejection, no details. See Christiansson patent.
Mathematical expression user interface using two-dimensional handwriting recognition, includes cursive text and math symbols recognition, two-dimensional parsing, symbolic and numeric solutions to equations, graphing. Compare with MIT work 1960's, xThink, etc. Includes video demo (youtube) for Department of Education.
Capacitive multi-touch touchscreen with proximity/hover sensing. Icons/keyboard grow in size as finger approaches. Asserts self-capacitance is single-touch only with ghosting (no multi-touch). Side-sensing (perimeter) to detect form of grasp, no bezel frame. Shows hover gestures (tilting flat hand over display/touchscreen). Compare with Rekimoto?
3D three dimensional manipulation using a flat 2D multi-touch surface: 3D images are shown below surface, with a simulated vertical "shadow" (cone shaped projection, narrowing as comes up) from virtual object up to touch surface. User can manipulate object by multi-touch on shadow at tabletop. Eliminates visual convergence problems, since fingers and shadow "surface" are on same plane.
Essay on "cooking" of experimental statistics (examples mostly in the social sciences) by phrasing questions after the collection of data: questions can be chosen/re-phrased for any particular data collection in such a way to appear to support almost any hypothesis. See also Ioannidis 2005.
Prosthetic hand/finger with conductive cover to work with capacitive touchscreens
Touchscreen user interface: touch on icon, slide off, perform function based on slide. Compare with Buxton keyboard?
Prototype two-hand/multi-touch user interface: static fingers of non-dominant hand on touchscreen change "mode" of gestures in dominant hand, similar to shift/function keys with mouse. Prototype uses colored band (tagged ring) on fingernails and overhead optical camera to identify fingers. Compare with Buxton on two-handed user interaction?
Tactile/haptic feedback on electrostatic multi-touch screen by driving X and Y lines for a particular point for feedback at frequencies above human detection (1000 Hz), beat frequency is perceptible (240Hz). Not highly localized, perhaps usable for keyboard images. Appears to be able to do only one location -- shadow effect?
Determine stylus tilt/orientation (and rotation) on capacitive touchscreen by measuring more than capacitance point on the stylus: stylus tip, and broken ring electrodes around the stylus just above the tip. Alternatives include to use accelerometer (gravity) or gyroscope to determine orientation to earth, combined with tilt sensor for the touch screen. Blesser/Pencept (tilt), Lee 1985 (multi-touch) listed in references: not Anoto.
Active stylus (GTCo style capacitive) combined with multi-touch capacitive touchscreen (like Rekimoto). If stylus is transmitter, both X and Y grid need sensors; if stylus is receiver, both X and Y grid need transmitters. Mentions encoding digital data in stylus signal (like M.I.T.). Says one advantage is that passive stylus need rounded tip (like finger), thus less precise than active stylus point.
Connect peripherals (wirelessly e.g. bluetooth) to a smartphone, use as desktop monitor. Compare with Ray Burns?
Video image of hands combined semi-transparently with computer image, so that user perceives hands as touching directly on image, without hands obscuring image. Any touchpad/tablet technology: also proximity sensing? Compare to Wellner, systems with rear-surface touching, 3D manipulation / virtual reality systems.
tangibles/physical widgets for use on capacitive multi-touch touchscreens; thin fiber optics bring the image up to the top of the widget, thin conductive wires down bring capacitive sensing up to the top of the widget. Fiber optics can brink image and sensing off edge of screen (bent over), to add a labelable button to the side of the touchscreen (lose a bit of display, but need not bee one-to-one.)
Combine multi-touch hand and stylus (e.g. able to sense orientation, type of grip) for gestural context based on behavioral context. Ignore unintentional touches (palm rejection) by context. Two-handed multi-model gestures: e.g. pinch gesture with while tucking pen in hand gives magnifying glass for drawing; touch/point with left hand while angling stylus above surface gives airbrush tool. Stylus grips: writing, tuck (stylus held like chopstick while writing hand is gesturing), palm (stylus held in fist), extension (point with a finger while tick or palm grip). Review of user-interface work on grip sensing, proximity, 3D position or orientation of stylus, hand gestures. Handing tablet to other user brings up per-user acetate drawing layer.
10 rules of thumb/observations for actual use of hand-held touchscreen smartphones. E.g. center of touch screen is preferred for both touching and viewing; fingers get in the way of viewing (e.g. icons and objects should be made visible in areas separate from where user will gestures to scroll); touch is imprecise (think of tolerances, not accuracy/errors); clickable/active objects must be visually distinct (underlined/outlined); UI must be designed for actual size of touchscreen from start, not shrunk down from tablet size display. File includes slides from separate presentation: "Make touch targets as large as possible", graphs of walking/standing/sitting accuracy. Graphs of tablet sensor inaccuracy (line traces).
Touchscreen/Touchpanel market, especially projected capacitance touchscreens, touchscreens integrated (in-cell) with displays. Vnet (Apple) uses same components for display and touchsensors: higher display resolution / frame rate lessens time for accurate touch measurements. "Tap-sensing" for force-sensitive touch detection a.k.a. Force Touch. Taptic sensing tactile/haptic feedback to simulated 3D key presses. N-trig force sensing stylus (optical shutter), Wacom electromagnetic (see Wacom file).
Fat-finger problem on touchscreens (especially smaller touchscreens on smartphones) unresolved: techniques in use include keystroke biometrics, use of stylus, NanoTouch using touch surface on rear of handheld display. Points out that stylus requires use of two hands, and is more subject to hand tremor.
Reference text on computer graphics: similar to Foley (also contributor). Described alpha blending as "coverage" (opacity/translucency/transparency). No index listing for BitBlt(!).
Teardown of Apple Pencil stylus for iPads. iPad touchscreen senses pencil, digitize rate higher at 240 Hz points/second vs. 120 Hz for finger/touch input. Similar to iPad Pencil by 53, which also had eraser at rear end (compare with Wang Freestyle). Capacitive tip very close to screen (parallax?). Movement-based (spring-loaded) force sensor.
Variation on pick-and-drop (cited): User can pick up a (simulated) object using multi-touch contacts on one touchscreen devices, and drop on another by touching with approximately same finger gesture position.
MyScript handwriting calculator -- see VisionObjects.
Marketing and product information on touchscreen display with capacitive multi-touch integrated: appears to be sheet of ITO within glass on front of display elements/LCD, matrix of scan electrodes within glass on rear.
Bookmark pages temporarily in electronic book by touching them with multiple fingers, like gesture of holding finger between pages. Multi-touch touchscreen.
User authentication by what log-in gesture user makes (in air/visual: also mentions proximity sensing on capacitive touchscreen) and how the user makes the gesture. Note: refers to user identification, not user authentication(?). Contrasts with signature verification. Compare with ASL gesture systems.
Physically extend capacitive touchscreen to off-screen margins (for extra buttons, sliders, etc.) using thin film with narrow conductors over send of touchscreen (examples are bottom margin). Example includes changing routing, spacing of lines to allow for wider spacing, circular (touchwheel) inputs, etc.
Commercial (Elo Touch Solutions) appeal that capacitive (projected) touchscreen technology will dominate, but not appropriate for all solutions. P-cap difficulty with any stylus (vs. resistive, optical), contaminant (e.g. water) rejection, high transparency (vs. SAW and optical), extremely large surfaces (resistive, SAW).
In-air gestures using both position and orientation of a hand-held device (WII?), for drawings/markings (graffiti) on image.
Electrostatic grid touchscreen, reduce power consumption by scanning fast/coarse until a touch is detected, then scan at full resolution. Compare with Pencept/Numonics course/fine scanning on electromagnetic tablets.
Biometric authentication of user by keyboard/keystroke dynamics. Notes that infrequent typing characters are hard to get statistics on (need large samples), but also have lower correlation on inter-keystroke timing. File includes presentation slides. See also Saltzer 1975 (not cited).
Proposal for combined fingerprint/multi-touch touchpad device: high resolution matrix, scanned at low resolution (every "N" trace lines) for touch input: for fingerprint input, when finger is found, re-scan just that area at high resolution (don't skip any traces). Can be implemented in-cell (with display) by sizing electrodes/sensor points as sub-pixel spacing.
Detect when two smartphones are tapped together on the screens from hover-touchscreen data plus accelerometers. Use gesture to start data transfer. Actual touch of one cellphone corner on other screen reported as hover not touch. (Because of contact area filtering, but the authors were mystified.)
Electrostatic sheet touchscreen, single-touch: User finger is active probe, signal input by wrist-strap: ratiometric measurement of signal picked up by resistive sheet. Appears very similar to Scriptel.
LG G3 touchscreen smartphone. Multiple gestures, beyond touch gestures. Turn phone face down to mute. Triple-tap, pinch, and corner-touch zoom gestures. Swipe/slide/flick gesture (synonymous). Hand-wave in front of camera to take picture, make fist to start timer delay. Touch assistant (control area) letter gestures: "C" to call, "W" to browser, "S" settings, "L" lock gestures. Flip gesture (of phone) to silence alarm. Proximity sensor: gesture of holding to ear. Handwriting recognition, virtual on-screen keyboard.
Interdigitated single-layer capacitive tablet: separate triangular traces. Position determined by which pair of traces (from opposite sides), and charge ratio (see). Variations include small teeth and indentations on sides of long triangles. Corrections for edge effects on bottom and top rows.
Array of (transparent) piezoresistive touch sensors for multi-touch force-measuring digitizer/touchscreen.
Stippled pattern on transparent film, optical video/camera in stylus to determine position based on pattern. Fluorescent dots for pattern, exemplary pattern is dots offset from grid. Assigned to NeoLab (see file). Compare with IBM display, Anoto?
Using gestures/shapes on a touchscreen to access a hyperlink. Cites to Agulnick.
Low-latency method for interpolating local maximum of touch points on multi-touch touchscreen ("heat map"), tracking touch points between frames. Shows diamond pattern of capacitive grid. Compare with Westerman, Rekimoto, Rympalski 1987?
Mechanical system for determining latency in user interface (touchscreen) by measuring time between input event and system response. Includes dragging latency.
Flexible touchscreen display sensor, matrix of small printed strain gauges. Compare with flexible/deformable input devices.
Transparent touchscreen with display, two users collaborating on opposite side, can switch touchscreen among transparent (direct communication), translucent/semi-transparent (collaboration and video), and opaque (displays for both users only). Not sure it comments on mirror-image/reversal between users. Compare with Whiteboard/collaboration systems.
Capacitive linear displacement and capacitive position sensors, primarily for industrial use: mostly linear sensors, not X/Y. Also eddy-current/inductive linear sensors. Includes TechNote L03-0020, Capacitive Sensor Operation and Optimization tutorial.
Single-layer capacitive touchscreen (not multi-touch) with one layer of electrode strips: X coordinate by which trace finger is near, Y coordinate by modeling resistance along trace as two resistors divided by a capacitor to ground where finger is along the trace.
Stylus for (touchscreen) with camera in tip, used to capture "texture" of physical object surface. Haptic feedback to simulated texture of a displayed object: or haptic feedback based on color. Compare Anoto, Rekimoto?
Stylus for (touchscreen) with sensor (force/contact/other), used to capture "texture" of physical object surface. Haptic feedback to simulated texture of a displayed object: or haptic feedback based on encoded pattern in surface to simulate physical text from visual textur. Compare Anoto, Rekimoto?
Shake or tilt a smartphone (shake/tilt gesture), as an indication to detect user mistakes on user interface, and adjust the user interface (move/shift/re-size icons).
General claim to finger flick gesture from dynamics of pressure matrix array touchscreen.
General claim to finger flick gesture from dynamics of pressure matrix array touchscreen.
General claim to finger flick gesture from dynamics of pressure matrix array touchscreen. Refers to "continuous grammars" for gesture changes without breaking touch: e.g. increasing pressure to scale size. Separate actions for taps by different fingers/thumbs.
General claim to finger flick gesture from dynamics of pressure matrix array touchscreen.
Slides of continuing workshop series NIME ("New Interfaces for Musical Expression", since 2001 CHI) on musical instrument interfaces. Historical discussion of theremin with capacitive proximity sensing. Haptic, touchscreen, force-feedback, force (pressure) sensing, torque/rotation/translation forces (side-ways force), proximity, Polhemus and Ascension flock-of-birds 3D sensors. TouchGlove and TouchStrip, TapTile FSR (force-sensitive resistor). Visual-input piano (optical/video sensor), ReacTable movable puck/object sensors. Kinect/ Hugh Le Caine keyboard touch sensor 1940s.
Haptic feedback signals for graphical objects on touchscreen.
Highlight/preview state of an application when first selected, then activate separately.
Reduce user interface latency on touchscreen input by having second (faster/low-latency) processing of user input to give immediate feedback based on static definitions of UI elements. Mentions low-latency system reducing events sent to high-latency application. Compare with PenWindows/PenPoint/Freestyle? Compare with Freestyle low-level inking, PenWindows inking and mouse event coalescing, PenPoint lowest-level inking.
Two Video presentations: one for conference, other as Webinar. "Turn any surface into touchpad": using Google glass or other optical head-mounted sensor, detect residual heat thermal signature on surface briefly after touched by human finger. Requires that optical camera respond to / distinguish infrared?
Prototype / proof of concept only, company (German) does Google glass / virtual reality applications.
File contains additional materials.
(in German) HyperBraille/BrailleDis 7200/9000 product information and technical sheet. 60x120 pin-matrix graphical display for the blind, includes touchscreen input with 1440 touch sensors. Also Modul D2 data sheet: 2x10 single-character Braille output, building block for BrailleDis with 2 touch sensors each (capacitive?).
Windows 8.1 Surface touchscreen: OneNote electronic ink note-taking. Gestures: Tap, Tap-and-hold (press-and-hold), tap-slide (tap-tap slide / tap and a half), pinch/stretch, rotate, slide/drag, swipe to select (quick small flick), swipe from edge, two-finger-tap (like 2nd mouse button), two-finger slide to scroll. Surface Pen (stylus) has rear button (compare Freestyle), right-click button (compare Freestyle): stylus proximity turns off touch (finger).
Corporate history page on Microsoft Windows. Interface Manager code name, announced as operating system 1983, released as Windows 1.0 1985. Windows 1.0 in 256Kb Ram. Windows 2.0/2.11 1987 Control Panel. Windows 3.0 operating system. Windows NT new operating system 1993. 32-bit. Mentions nightly build/stress tests in development. Very brief mention of Tablet PC, none of Pen Windows: Windows Touch on Windows 7.
Gestures and use on Microsoft Surface touchpad (on keyboard, not display touchscreen). One-finger tap, two-finger tap, drag, left-click (on touchpad button) and drag, tap-tap-drag (Tap and a half), scroll, Pinch/Stretch/zoom gestures.
Fine Arts faculty profile for Slavko Milekic. Artistic presentations use gaze-tracking, touchscreen input. Mentions KiddyFace for art exploration with children.
MobilePoint. forms-based pen computing applications for field sales. SalesTrak Mobile for iPad. Information stored locally (does not require live internet connection). Medical sales features. Originally a PenPoint port to re-boot on PenWindows hardware.
Single point (one fingertip) haptic/tactile in-air feedback using focused ultrasound, combined with half-mirror 3D three dimensional display (Aerial Imaging Plate API ASUKANET commercial unit). 40 kHz signal has focal spot about size of a fingertip (one wavelength), modulated at 100 Hz to produce sensation. Compare with UltraHapics project at U. Bristol.
Tablet/hand-held computer (smartphone) with second touch screen on the back/rear side, with rotation/orientation sensor. If held vertically, only bottom half of rear touchscreen is active.
Apple iPhone Force Touch: force (pressure) sensing on touchpad or touchscreen, used on Apple Watch and Apple iPhone harder tap == right click/dwell gesture, or harder touch == faster scroll. Also on MacBook touchpad force sensing.
Handwriting and shape recognition with applications components for SmartPHones: MyScript formerly VisionObjects (see file). Automatic indexing of ink notes (compare with PenScheduler?). Autocompletion from spelling dictionary. Developer documentation: single-character, line, multi-line, cursive, music notation recognition input widgets. Includes announcement of integration with NeoLAB smart pen, similar to Anoto?
Sides of touchscreen mounted on small pillars, can detect horizontal (and vertical) shear forces. User interface to deform virtual objects. Compare to Margaret Minsky?
Tutorial/Overview on LabView system and graphical programming language, 2014. Focuses on programming of virtual lab instruments in LabView VirtualBench. Describes VirtualBench APIs: the graphical programming interfaces between LabView virtual instrument (VI) objects.
Product information on Neo SmartPen: patterned paper in special notebooks, optical camera in pen to recognize position (and which paper form) from "Ncode" pattern. Include NeoNote app software for IOS and Android smartphone to enter handwriting notes (to EverNote?). press-and-hold popup graphical menus to select ink color, width. Integration with voice notes (compare with Wang Freestyle?). CogniVue optical camera hardware. See also MyScript (formerly VisionObjects) handwriting and shape recognition. Compare with Anoto, IBM pattern?
Historical notes on an early evaluation of PenPoint touchscreen/tablet user interface. Superiority of stylus over finger from accuracy (tap/read asymmetry), PenPoint generic commands with unified command syntax: no verb-then-noun or noun-then-verb two steps, draw gesture (e.g. squiggle/X/delete) directly on object. Gestures could also determine scope/which operand: X to delete character, rubout/squiggle to delete word.
Ocular: Projected capacitive / electrostatic touch panel tablets: Crystal Touch / Edge. Thin glass. Optically-bonded glass cover optional instead of air gap between layers. Shows flush and bezel mounting. Includes multi-touch demonstration software with finger tracking. Recommends touch areas (accuracy?) of 10 mm plus spacing. Mentions two-finger pinch/zoom without multitouch.
Optical/camera detection of hand gestures in a smartphone, for controlling a projector built into the phone (or other device). Waving hand left/right to scroll. Basically only detecting gross motion of any kind, not discrete gestures.
When use selects something in a hierarchical or list view, (e.g. multiple columns of details), show the item(s) of interest fully, make the others smaller or put their windows/containers behind the objects of interest. Compare with Magic Lens, Slate hierarchy?
Accelerometer stylus (compare with Hugh Crane), records or transmit movement data (compare with Anoto?). States that user can write on any surface (including whiteboard), or in the air. Refers to accumulated positioning error, stylus can be reset to a zero point or re-calibrated at anytime: touch four positions to be four corners to match mapping to a display. Primarily a thought-experiment?
Design project for secure CPU processor: programs in specific "trusted" area of memory protected from modification by software in other areas. Compare with DEP data execution protection, memory mapping, Yee protected processor? Cites to Saltzer 75 for seminal security work
Penclic Mouse B3: Mechanical roller mouse in form of pen with roller in tip, compare with PenMouse from 1980's (in collection). Wireless bluetooth interface. www.penclic.se product literature in file includes cable versions. Multiple barrel buttons for left/right mouse button, center roller, page up/down. Illustrations seem to imply detection of rotation of stylus/pen.
Penclic external touchpad: with four buttons for left/right click, forward/back keys (for browser), additional physical spin wheel /scroll wheel for scrolling.
Empirical study of bullshit statements: receptivity to accepting bullshit related to subject's profession to religiosity, supernatural beliefs, alternative medicine, intuitive cognitive style. Tendency to accept statements as true prior to analysis to determine they are false.
User-interface combining eye-tracking (gaze) with multi-touch gestures on touchscreen: Gaze/eye-tracking selects, touch manipulates. Gestures can be made at any location on objects selected by gaze "dwell". compare with Wellner etc. large wall displays. Problems noted include multiple-select (essentially lose multi-touch for select), "muscle memory" gestures (activating PIN keypad) etc. where user is not gazing at the object to be manipulated.
Optical sensing touchscreen: optical emitters at sides direct beams within glass to multiple optical sensors spaced on opposite side, touch on surface of glass disturbs (reduces) frustrates total internal reflection (FTID). Comparision of disruption of multiple beams with pre-analyzed disruptions by known reference touches, determine multi-touch inputs. See Rapt Touch file (assignee). Compare with FlatFrog?
Resistive/conductive films, 200-micrometer-sized metal mesh in polyester/PET film substrate, alternative to ITO indium tin oxide, moldable to curved surfaces. "Touchscreens mit transparenten, flexiblen und leitungsfäigen Folien"
User study of BrailleDis 7200 tactile pin-matrix display for the blind. User interface has multiple controls: on-screen sliders and buttons, function keys, cursor keys, separate touchpad. Function button to switch to gesture input (vs. "view" mode where user scans display). Separate touchpad not as good as touchscreen input.
"Touchless Gesture control": Passive infra-red optical video sensor for detecting hand and body gestures: contrasted with touchscreen and with Microsoft WII.
Optical touchscreen using radial pattern of internally-reflected light beams through glass layer. See Piot reference for details. Also see video file, same web site.
Very small touchpad keyboard for watch-sized devices: long-tapping vs. short-tapping: holding finger on keyboard for longer time with zooming UI display allows for more precise input of characters, system then knows not to do word-substitution/spelling correction.
Direct pressure-sensing capacitive multi-touch touchscreen: elastomeric dielectric material layer (silicone) on front, measure force/pressure by degree of coupling as user presses fingertip into elastomeric layer. also proximity sensing.
Capacitive multi-touch touchscreen with multiple layers, traces/guards run on additional layers. Reduction of parasitic capacitance, can detect proximity at greater distance, less problematic at edges or near other components/objects.
ITO-replacement, stretched carbon monofilament film on polycarbonate substrate. Largr-format tabletop/touchscreen, multi-touch (up to 10 touches) projected capacitance.
3-D display (of sorts): flat display suspended from movable support arm on ceiling, user can move display in 5 DOF degrees of freedom (X, Y, Z, pitch, yaw -- no roll) to see views of volumetric images. More importantly, display can move autonomously without user's hands: e.g. user requests view of something (e.g. a particular portion by name of a 3D image of a brain), display moves autonomously to position and orientation to display view. Uses in-air gestures (e.g. in-air pinch gesture to "pick up" display so that it is moved in direction of hand motion.
Long-term project review (back to 2001) for thin flexible electronic skin E-skin touch sensor, proposed uses are robotic sensing, medical instrumentation. Sensor skin applied to throat to recognize speech by muscular activity. Plastic substrate, TFT touch sensors. Shows pressure profile from array of sensors.
Tactile/haptic electrosensory display. AC-modulated signal coupled capacitively to fingertips at discreet pads, sensation is direct Coulomb force capacitive to Pacinian mechanoreceptor nerve cells in fingertip, not physical vibration or electrostatic friction.
Matrix pressure-sensing tablet, piezoresistive grid, minimum sensing grid resolution approx 1.6 mm. Matrix can be flexible, and also stretchable (elastomeric). Targeted as tactile sensor for mechanical testing, measurement of fit (of chairs, mattresses, etc.).
Parallel/simultaneous touch (multi-touch) and stylus sensors on a touchscreen display, synchronized to refresh signal for display.
in-air/proximity gestures on smartphone using regular smartphone camera on front. States advantages for mobile use, hands-free, display screen not obscured. Continuous recognition of pinch/zoom gesture motions of hand (move in and out, etc.). Compare hand/gesture detection with Wellner? Mentions touch+gestures (see files).
"floating touch" / hover on touchscreen. Technical description describes use of both mutual capacitance (for position) and self capacitance (for hover height) (???), hover/proximity gestures cannot be multi-touch (???). Cypress Technologies.. Android MotionEvent method onHoverEvent.
Resistive (conductive) glove for use with capacitive touchscreen. Conductive yarn twisted around other yarn.
Textbook/reference on Perl programming language
"Zero-latency" capacitive multi-touch: reduce display latency responding to touch input by up to 33 msec (two video frames). Inking, UI response done at interrupt level: states does not use predictive input. Founding staff from U. Toronto. See also Freestyle inking, PenWindows and PenPoint? See also Bill Buxton? Andries van Dam also on advisory board: Forlines, Wigdon, Leigh (from DiamondTouch), Jota
micro-fluidics channels in thin layer, can be inflated to create physical buttons on transparent touchscreen as haptic display.
Performance comparison of multiple/various systems typewriting / typing dynamics as biometric authentication . Typing style (hunt-and-peck, touch-typing, etc.), keypress/keystroke latency/timing, etc. using data from only one of the two hands. See Salzer early reference.
Review of smartphones and tablets that come with a stylus: Microsoft Surface Pro with N-trig stylus digitizer, Samsung Galaxy Note 1.0 with electromagnetic (?) stylus. Yoga Tablet 2 (Lenovoa) use any *metallic* object for stylus.
3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick?) for pan, zoom rotate.
3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick?) for pan, zoom rotate. Five near-mouse function keys, use short press (tap) and long press (press-and-hold) for 10 functions. Function keys automatically redefine function according to application (default function of OS?). Keyboard modifier keys (shift, alt, ctl) that are often used with mouse input as qualifiers/modifiers.
3D three-dimensional "mouse" input device Sales brochure (?) on CAD services and software: Third-party Product Data Management PDM tools for collaborative 3D CAD design work: GrabCAD Workbench, Tinkercad, Sketchfab, WikiSpaces, HackPad, etc.
Data Glove for industrial applications with gyroscopic sensor, accelerometer, contact detection sensor on index finger: compare with Micorosoft WII controller? Motion sensors used for gestures, contact sensor and tag reader (RFID?) used to recognize labels on equipment. Intended as wearable data entry device.
transparent capacitive-grid (multi-touch) touchpad film, overlaid on keyboard. "Machine learning" to distinguish moving touchpad input and keyboard typing input. Call capacitive sensing grid a capacitance-to-digital converter. Cites to Rekimoto ThumbSense, but not to multitouch tabletop.
Wayne Westerman, John Elias named to National Academy of Inventors: multi-touch interfaces with scrolling, finger tracking and gesture recognition. John Elias multitouch design started 1995.
Wall-sized display with tactile feedback (vibration)
Transmit additional information through existing capacitive/electrostatic touchscreen: circuit in signet ring on user's hand generates signal that can be picked up by existing touchscreen electronics at low data rate (5..10 bps). Can also be used to identify multiple styli/styluses, multiple users, etc.
Review of produced/exhibitors at Display Week exposition: embedded-touch and in-cell touchscreens and sensors, integrated with display. Reverse-side electrostatic/P-CAP; single-layer mutual-capacitance using "caterpillar pattern"two-element grids with crossovers/bridges; "Pixel Eyes" (JDI vendor); optical fingerprint sensor, very high resolution, LED pixels used for touch sensing. Water resistance: does not affect self-capacitance sensors, only mutual-capacitance. Electromagnetic resonance touch battery-less unpowered stylus from Hanvon: compare with Walcom? ITO replacements carbon nanotubes, silver nanowires, silicon-doped zinc, PET PoliyIC film. photolithography printing of mesh. Resistive-film multi-touch using 12 mm sensor pads.
Comprehensive tutorial of touchscreen technologies as of 2014. Emphasis on projected capacitance (electrostatic). Electrode patterns for grid, including single-layer grid geometries. Notes that pressure/force sensing is actually contact area. Optical coatings, ITO-Replacement materials, embedded sensors, Stylus. Cites to Buxton. Illustrations from Apple patent, no mention of FingerWorks or Rekimoto. Resistive multi-touch by dividing into small finger-sized squares. Cites to JazzMutant as first multi-touch product.
Text-selection gesture brings up text selection with movable handles (instead of start/end selection gestures). Handle can be a "phantom" handle for text position off-screen. Cites to Hullender gestures.
Encryption/DRM application platform (with API) for protecting multimedia (especially Video) documents and content. Uses secure hardware for keyboxes to store private PKI keys. Google DRM system. See also ContentGuard.
Reduce user interface latency on touchscreen input by having second (faster/low-latency) processing of user input to give immediate feedback based on static definitions of UI elements. Mentions low-latency system reducing events sent to high-latency application. Compare with PenWindows/PenPoint/Freestyle? Tactual Labs: Compare with Freestyle low-level inking, PenWindows inking and mouse event coalescing, PenPoint lowest-level inking.
OpenGL: 2D and 3D vector graphics library, includes alpha blending. API specification for access to hardware.
BitBlt binary block transfer operator/operation.
halftone image typically 300 dpi or 65 lpi resolution on laser printer, lower on display
Standard structure of an application (look-and-feel / GUI). OS-specific examples (MacApp), Cocoa (iPad), also cross-platform (Qt, wxWidgets, etc.), NetBeans, KDE.
General information on Multics OS "Multiplexed Information and Computing Services": access control lists on files/objects, all files memory-mapped.
Generalized concept for application framework: includes inversion of control, default behavior, extensibility, non-modifiable framework code
History of UNIX System V through 2014. Access Control Lists ACL 1993(?), SVR 4.1 / attached usage rights.
History of Microsoft Windows releases and versions. Windows 9X/ME operating system. Early versions used MS/DOS for file system, Windows had own device drivers, (cooperative) multitasking, segment-based software virtual memory: compare Android on Linux. Refers to earliest Windows as "operating environment", not OS.
PKI signatures and timestamps to authenticate software: Cites to Lotus Notes Release 1.0. Alternative to CA certificates is to embed public key in installer.
Power Management: APM, ACPI: full on, APM enabled, APM standby, APM suspend/hibernate, Off
RPC / RMI: remote method invocation. Early history in Arpanet. Summary listing of protocols and standards.
Inkwell/Ink handwriting recognition software on Mac OS X, originated in Newton. Not used in iPad/iPhone/iPod. File contains additional materials. Press-and-hold to switch from inking/handwriting to mouse mode on touchscreen.
History of Amazon Kindle e-book reader: EPUB and AZW formats, reflective and backlit displays, keyboard.
WML: Reduced HTML version for us on small displays with low bandwidth (e.g. PalmPads, etc.)
Dell Axim PDA / Pocket PC running Windows CE.
Apple iPad Mini tablet, 2012. iBooks e-book application optional. Internet access required for set-up.
Pointer/reference/dereferencing: pointer data type in PL/I, 1965. Formal definition of pointer arithmetic, abstract reference. Autorelative, based, doubly-dereference/doubly-indirect pointers.
Low-end Palm touchscreen PDA: m100, m105, m125, m130. Grayscale or color displays.
Virtual Keyboard: soft keyboard, on-screen keyboard. Mentions chord keyboards, latkey.com multi-lingual keyboards.
Study of resolution/accuracy of haptic feedback using ultrasonic acoustic radiation pressure. Discussion of various nerve sensors: Pacinian corpuscles (vibration); mechanoreceptors have higher acuity. Hand-gesture sensors and tactile feedback mounted on user's hand.
Protective cover/shield screen protector for a touchscreen, suspended by air gap over touchscreen (when not being pressed) so as not to have bubbles, Newton rings, other optical problems.
Experimental system using 3D "hover" information on fingertip to predict point of contact, and signal touchdown with less latency. Microsoft Surface (Samsung SUR40 tabletop / PixelSense) with 3D camera tracking system of marked fingertips. See other Wigdor, Forlines, Jota references.
Haptic feedback (simulated traction/friction/repulsive forces) on stylus, using asymmetric vibrators for any X/Y direction. Cites to Traxion by Rekimoto. Suggests using force to guide/teach user, rather than to simulate feel of virtual objects (brush, etc.), due to limited angular accuracy of prototype.
Optical (FTIR?) touchscreen sensor using linear waveguides and linear light bar sources in a rectilinear grid/matrix. Touch contact sensed by effects on light dispersion at intersection points. (Compare with Rekimoto?). Overview has good overview of resistive, capacitive (trans-capacitance and mutual-capacitance), SAW acoustic, acoustic pulse, infrared/optical beam touchscreen sensors (no FTIR?).
Real-time and asynchronous (recorded) document review system including electronic ink annotations, synchronized voice, pointing, gestures. Hovering places "highlighter" color for pointing. Cites to Wang Freestyle: Freestyle could also do pointing gestures via "hover". Users spontaneously allocated particular space for comments: bottom of page for general comments on page, to side of text for comments on particular text. Compare with Anoto SmartPen? Space can be expanded for additional ink (+ voice) annotations: compare with van Raamsdonck?
Interesting study of abilities of service dogs to learn to use a touchscreen interface: applications include calling emergency numbers, etc. Also found that ability to learn and perform this task ("drive") predictive of ability to focus and work as guide guide dog (to save training expense on dogs without enough "drive"). Authors appeared confused about multi-touch vs. single-touch touchscreens: switched to IR/LED optical touch screen (not identified) because saliva and moisture from dog's snout (one dog spontaneously used paw) affected capacitive touchscreen.
Proximity sensor used for hand gestures on smartphone: additional antennas pick up reflect GSM signal from smartphone itself, hand proximity affects signals received by multipath reflection/propagation. Recognize in-air swipe gestures, proximity on different edges and in front.
Authenticate user using biometric device (fingerprint reader) containing cryptographic keys, API access to device once authenticated: fingerprint scanner? See MAZ Lenovo.
biblio13 (Prev page)
(Next page) biblioxx
This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at firstname.lastname@example.org.