This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015.
I can be contacted at firstname.lastname@example.org.
History of Pen and Gesture Computing:
Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers
References from the approximate years 1994 to 1995.
Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement:
Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward
Most of these are in my personal collection, either as physical items or electronic files.
Copyright © Mon Oct 5 23:56:37 EDT 2015.
Mirrors of this page can be found at:
biblio93 (Prev page)
(Next page) biblio98
Vision of future technology. Shows personal hand-held video communication (similar to Skype) with pen touchscreen interface.
Implementation of a slot object: citations back to 1982
Active Desk whiteboard / collaborative drawing space prototype using large Scriptel digitizer tablet and rear-projection display. Compare with Microsoft Surface, Digital Desk
Add gestures (marks) to point-and-click direct manipulation Motif widgets without modifying Motif implementation: filter recognizeds gestures, callback hooks into mouse events, calls existing API function in widget.
Describes using a pen stylus for entering commands via character recognition / gestures into a computer, recognizing and executing command after out-of-proximity/end-of-stroke/time-out. Compare with Ledeen, which did not have gestures made on the display. Specific reference to first, second and third gestures in a sequence for a compound gesture (multiple flicks, multiple taps, etc.). Specific reference to direction of motion of the gesture. Animated marquee on selection after press-and-hold. Describes feedback to user (idealized form of gesture displayed, or question mark in a circle) on what gesture recognized. If gesture not appropriate in context, error returned (displayed). Proximity sensing for termination of gesture, highlighting of active objects (compare with tooltips)
Slider/selector/scrollbar for scrolling through long alphanumeric sorted lists (e.g. rolodex or dictionary): variable-speed scrolling (acceleration) if slider thumb moved more quickly. Acceleration not work well, especially since user could just tap on alphabetical index (linear alphabetically) next to scroll bar. Also micrometer (fine positioning) mode by moving mouse perpendicularly to scroll motion: confusing to users.
Optical (camera) hand tracking system for smart desk: track hand, multiple fingertips, finger joints.
IBM Simon: up/down physical buttons for volume on cell phone switched to contrast-adjustment mode either by pressing both buttons together, or tapping on a particular spot on the touchscreen.
IBM Simon: cellphone/smartphone with touchscreen, can show screens in both landscape and portrait orientation based on user on-screen rotation button.
Polhemus 3D digitizer/tablet position sensor: uses pulsed DC magnetic fields, so is affected by nearby metallic or ferrous objects. Compare with Ascension Techology?
Select new color from a palette, drag onto a UI element to change its color in OS/2. Dynamic themes/scheme manager via GUI.
File also contains EP published patent application 0 591 571 A1.
Video of Apple TabletBook, prototype Apple MacIntosh slate-format tablet computer, touchscreen on display.
Programming reference for QuickDraw graphic on MacIntosh. Regions or maskBits used for masking -- any number of closed regions defined in drawing coordinates vs. bitmap coordinates. modes (RasterOp) on copyBits (BitBlt) include SrcOr, srcXor, srcCopy. Automatic dithering and highlight (background color replacement) in modes. Modes blend (average), sum (addPin, addOver)
Apple Newton support for wireless communications, not on a single-vendor network
User-interface guidlines for Apple Macintosh: compare with similar publications for Microsoft Windows. Mentions stylus pen (tablet) as equivalent pointing input device. Pointers/cursors for scroll bars and controls (arrow), drawing and re-sizing (crosshairs), selecting and inserting text (I-beam), selecting fields in spreadsheet/array (plus sign), wait (wristwatch clock). Pressing == press and hold? References to academic UI literature. Cites to Buxton, Schneiderman for direct manipulation. Also Cites MinskyMR84 (gestures).
Apple prototype products, publicly known but not marketed: W.A.L.T. Wizzy Active Lifestyle Telephone with stylus -- compare with IBM Simon (also developed with BellSouth); Apple videophone PDA; Applel PenLite / Macintosh PowerBook Duo Tablet computer (full tablet/touchscreen computer, not PDA)
Video demonstration of Apple Newton user interface, including "heavy mark" gesture. Video in electronic files: transcript fetched from guidebookgallery.org 2010. Text file has complete transcription of text. Zigzag gesture to rubout (gesture). Copy-and-paste by moving text to edge of (small) screen. press-and-hold gesture on Newton "heavy mark", with highlight. Press-and-hold highlight is to move text (12:50), different drag and tap-drag gestures for move and for copy. At 12:20 shows rub-out gesture recognized in inking/writing mode. 12:40 pop-up virtual keyboard.
File transfer / connection utility between Apple Newton and Apple MacIntosh. Also calendar Date Book synchronization. Network connection? Parallel cable?
Apple Newton: describes various gestures and UI features, including "heavy mark" selection gesture: press-and-hold in text? Describes rotate button to change screen orientation
Apple Newton: whitepaper about product concepts for Apple Newton: Apple Newton fax/phone (desktop) with notetaking, Newton Map, fold-out three panel touchscreen pen computer, Newton touchscreen wristwatch (larger panel -- wrist-mounted computer), Newton classroom whiteboard, Newton family message center (Message pad hanging on refrigerator)
Indexing (tree based) of electronic-ink names of objects by applying hidden Markov Models HMM to match the handwritten characters one at a time, take best match. Matching handles mix of hand-printing and cursive forms of a name. Compare HMM with actual recognition?
Automatic indexing of electronic ink notes doing approximate ink matching (AIM) instead of recognition. One application is indexing of multimedia objects.
Second volume in "Graphic Gems", collection of papers: 2D Geometry and Algorithms, Image Processing, Frame Buffer Techniques, 3-D Geometry and Algorithms, Ray Tracing (transparent or translucent objects), Radiosity, Matrix Techniques, Numerical and Programming Techniques, Curves and Surfaces
Cited in MyersB96: ITS from IBM Research as automatic generation/generator tool for user interfaces.
Contains Case study C: Wang Freestyle, pen-based voice/electronic-ink annotation system. Describes substantial public interest and acclaim when Freestyle released in 1988 (citations)
See individual articles
Essay on legal, cultural, and other (non-technical) aspects of copyright protection and intellectual property, in the contest of D.R.M. digital rights management.
Transparent menus/windows in graphical user interface, does not block view of elements on screen underneath
To see windows underneath/obscured by other windows, user selects a mode whereby all windows become transparent: only the title bar and the frames are drawn. User can click on title bar (or frame) to bring that windows to the top, and transparency dropped.
Infinite resolution multimedia sketchpad: infinite resolution display windowed into smaller display area: parallel lazy loading, only load/render data that is current viewable: example is browsing photos.
Alternative graphical user interface to icons menus and windows, show small snapshots / postage-stamps of simulated desk objects. Directory browser is array of small thumbnail images of pages/documents.
Multiple scale / multi-scale data visualization user interface: example is directory browser, with thumbnails inside of thumbnails and multiple levels
Zoomable user interface, example is "world view" of documents, similar to thumbnail. Compare with Telesis "world view".
Multiple graphical representations of data, constraint-based graphical editing
Screenshots of Microsoft touchscreen PDA user interface, mostly from built-in tutorial. On-screen keyboard by touch of stylus to bottom-of-screen menu or an icon. Handwriting in boxed text fields: tap on field to bring up alternate word or character dictionary pick-list. Electronic ink stored as (erasable) bitmap image? Navigation tabs to right of screen -- compare with PenPoint? Screen rotation / orientation portrait/landscape chosen manually by user, similar to PenWindows dialog.
See-through transparent/translucent tools can be placed over an object without obscuring objects under the tool. One option is a gesture-interpreting tool. (Compare with gesture-buttons.) Two-handed input (trackball and mouse cursor: multitouch?) trackball moves a virtual transparent layer, such as a transparent selection/framing tool and a mouse click to make an action happen.
Video demonstration to accompany paper of same title. At about 2:00 shows dragging a "Fill Color" menu with transparent parts over an image. Dual-pane clipboard for picking up diverse multiple objects.
Video demonstration to accompany paper of same title. At about 0:25 shows a tool sheet (menu with buttons) with transparent sections being dragged over an image. Transparent pie menus / marking menu gestures. Unistroke gestures on a tool: compare with gesture buttons. 3:10 gesture interpreter tool / handwriting pad (Goldberg unistroke recognition). Mnemonic gestures: e.g. "C" for copy, "T" for translate/move --
Video demonstration. Shows tool window with frame and buttons (but most is transparent) being dragged over a map, and certain features of the map are altered (pixels combined?) Also, at about 0:59 shows translucent overlaying of two tool windows.
Video demonstration. At about 0:50 shows window tool with circular "view" area that changes color of pixels underneath (combined pixels), with part of tool transparent, also with buttons and controls in the window.
Mathematical process for aligning/matching portions of surface from multiple views from a single camera. Compare with structure-through-motion.
PIM/Diary that keeps change/delta diff record of changed entries so that previous entries can be reconstructed, and automatic monitoring for dates and other phrases to prompt user to add to diary. Compare to marking menus/keyboard?
Mentions Scriptel WriteTouch front-mount digitizer for Windows95 and Windows for Pen Computing: cites Windows 95 as having better pen-computing support than Windows for Pen Computing 1.0, although tablet drivers are not included. Norand PEN*KEY6100 vertical market pen tablet device.
Tutorial on alpha blending / compositing / matting /overlaying (synonyms), mentions anti-aliasing as one use, primarily "over" operator for bitblt/raster-op from Porter and Duff.
Tutorial on alpha blending / compositing / matting /overlaying (synonyms). Focuses on optimizations of integer arithmetic for 8 and 16 bit integer operations on 8-bit (32-bit) RGBA values, rounding and quantization errors.
Capacitive matrix grid digitizer: objects near intersections of wires affect capacitance between wires. Gets area measurement of adjacent intersection points, computes centroid/peak from adjacent (independent) sensors nodes. Used for mouse and for touch keypad in one device. Compressibility of fingertip to infer pressure (force). multi-touch capable? Compare to Buxton 1985 multi-touch capacitive tablet.
See also Radio Drum, Boie 1984 paper on capacitive sensors, references on "capacitive moments".
Sealing for SAW touchscreen tablet: see citations for actual surface acoustic wave touchscreen technologies
Elographics IntelliTouch Systems Manual, 1988
Recommends PenRight! Pro as leading pen-centric development environment, Windows for Pen Computing lacks pen-centric features, non-pen-centric development tools are unfeasible. No mention of Newton or PenPoint.
Electronic ink text editor, electronic ink automatically segmented based on spacing without character recognition, gestures include lasso and brackets to select, arrow to move text, double-ended arrow to copy, caret to insert, X to delete. Compare with van Raamsdonck.
Smartphone / communicator with video camera and display: camera is mechanically steerable
tablet-free stylus/pen with tip-force (pressure) sensor and accellerometers in tip, handwriting recognition (actually character recognition). Compare with Crane/SRI accellerometer pen (not cited). Uses special delimiter characters/gestures (e.g. right/left corners) to mark start/stop of recognition.
Web browser, pages can be stacked in multiple decks on screen: no 3-D effect on desks -- compare with Freestyle documents. Suggested use is keeping related web pages together in a single deck.
Optical digitizer tablet: Note similarity to Anoto: printed surface of pattern cells, making use of sub-patterns. Refers also to a "single-pixel" sensor stylus.
PenRight cross-platform development environment for pen computing: PenRight! for Windows create Windows and DOS applications for PDAs, tablets, handhelds, etc. from single source code
Haptic feedback stylus/pen (one application is simulating surgical feel). Stylus tip connects to direct-coupled (no gears) actuators, 3 DOF degrees of freedom (not 6). Special attent to open-loop control functions.
See also http://www.billbuxton.com: User flips keyboard top over to expose touchscreen: touch tablet is multi-touch sensitive and pressure (force) sensitive, with virtual sliders and further touch elements.
Two overlapping operations by user as same time using pointing input in both hands. Describes scrolling gestures: smooth scrolling as touch-slide-release, page turning with touch-rapid flick-release, and jumping to scrollbar touch location on a touch surface. Also mentions wireless touch tablet PARCtab.
See also http://www.billbuxton.com
Graphical on-screen keyboard using gestures for modifier keys such as shift and control: e.g. tap "a" key for lower-case a, upward flick for Upper case, right flick gesture for Control-a, right-upward angle gesture for control-shift-A, etc. Mentions interchangibility of other types of input devices: mouse, tablet, trackball, etc. press-and-wait / PenStill / press-and-hold for marking menu, gesture and wait at end of mark also for multiple heirarchy of menus. Four events: PenDown, PenDrag, PenUp, PenStill (with no movement for a period of time).
haptic input overview.
Date approximate based on dates of most references, this draft obtained 2002.
Video presentation of two-handed user interface with tool in non-dominant hand, tablet stylus in dominant hand. Companion video to "A taxonomy of see-through tools" and "Toolglass and magic lenses".
Object-based graphical development system with polymorphic inheritance
Collection of tutorials on information/computer security. Personnel and administrative policies, legal aspects. Long treatment of cryptography/cryptology/PKI, basic techniques of virus/worm including hooking. Access control primarily about physical access / authentication of users.
Handwriting recognition that let use display pop-up list of alternative recognition results for words (from spelling dictionary) by entering a gesture to bring it up, user can pick appropriate word, or use virtual keyboard to type in correct word.
Rotate an object with a stylus by means of a center axle mark, and a handle used to rotate.
On pen computer or tablet, pop-up list of alternative handwriting recognition results, user can pick from list or enter word via virtual keyboard. Figures show pop-up keyboard of Apple Newton. Correction gestures (insert, delete gesture, etc.). Refers to keyboard gesture (tap on keyboard image), correction gesture (tap on menu of alternative words).
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Gestures on buttons or command-input areas to edit electronic-ink notes: notes not necessarily ink.
Few if any references before. Compare with van Raamsdonk, Pencept, and CIC.
Automatic re-flow / formatting of paragraphs from handwritten (or "pointing" input): compare to van Raamsdonk.
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Few references before 1990 Compare with van Raamsdonk, Pencept, CIC
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC
Shows graphical/touchscreen slide switches and toggles, three-state "secure switch" toggle. Compare to Apple slide-to-unlock gesture?
User Action Notation for user interface objects. Example: Secure switch graphical object for preventing accidental manipulation of a device under computer control, such as master power switch for factory machinery. Compare to Apple slide-to-unlock gesture?
Personal note on EO with PenPoint: lists of collected EO/PenPoint materials. EO 440 hardware specifications. EO based in UK?
Compare Pen-Windows and CIC Handwriter recognizers: the CIC handwriting recognizer performed better, both with and without constraints.
GRIGRI (French slang for "scribble") system using pen-computing gesture recognition in user interface for air traffic control ATC. Modeled on flight strips for air traffic control. See also "Digistrips". Numeric entries for flight numbers via menus. Cites PenPoint. Lists fifteen single-stroke gestures, some of which are alphabetic: set warning, lasso (select), C, erase/pigtail gesture, undo alpha gesture, h change heading, etc.. Mentions that pen-based computing is bad for large ATC display screens (arm fatigue). Refers to gesture input as "mark-based input". Uses semi-transparent windows /button so as not to obscure radar image underneath window. See Rockstar Samsung.
Summary and overview of physical types of targets used in 3D capture: e.g. LED's, patterned crosses, reflective films, etc. No mention of using features of existing objects.
Portable slate pen-computer using touch screen (not proximity digitizer): Microslate, Inc. of Canada. Describes having an automatic application generator. Refers to drop-down pick list as a "library" of answers for touch-screen selection.
The Grunt: Texas Microsystems 3.5 pound Windows for Pen Computing portable computer using MicroTouch touch-screen: pen or special pen/stylus.
Pen-based computers / pen computers useful in data collecting with UPS, engineering maintenance on ships, of immediate access to data (stored in device) and faster to input test or delivery data. Mentions that generally no or little use of handwriting recognition, data entry by picking on pre-designed forms, adjusting sliders, sketches. PoquetPad hand-held pen computer, CruisePAD separate tablet/display for wireless access to PC within 100 meters
Transparent overlay windows in graphical user interface, pixel blending
Instead of password, user inputs a pattern of touches or events for authentication: slide-to-unlock?
Architectural overview of the PenWindows / Windows for Pen Computing system from Microsoft: hedit and bedit controls, Pen Extensions, normal and inking mode, RC handwriting recognition manager, PenWin.dll
LabView manuals from early 1990's for visual programming language. Shows slide switches, manuals for sale?
Suggest "X" windows as solution to lack of control over exact appearance of HTML documents: Pixel-level control. Extends to remote execution (X-client/server). Suggest LBX (Low Bandwidth X) protocol to reduce latency/bandwidth problems. Similar presenation to "X Over the Web", WWW3, Darmstadt, Germany april 1995.
Open Look windows management system / user interface. Drag-Right-Distance to select sub-menu. MoveThreshold, MouseDamping system properties. Compare with press-and-hold
Electrostatic / capacitive digitizer from Scriptel, can sense finger input as well as stylus
Donald Drumm "Orb" project from Wang Freestyle: pointing device for accessibility, uses head-mounted tilt sensor to push cursor position, simple voice detection circuit that detects fricatives "s" sound versus vocalized sounds.
Standardization functional specification of Microsoft Windows 3.1 application programming interface. WS_EX_TRANSPARENT creates transparent window. Also GetBkMode TRANSPARENT parameter for drawing. Transparent windows are included in windows hit testing, transparent windows can be buttons/scrollbars. See Rockstar Samsung.
LabVIEW graphical programming language.
Refers to mapping of events to mouse buttons in user interface. Compare to press-and-hold.
Contains description of touchscreen implementation of mouse menus for DOS application which are not "mouse-aware". See also Microsoft documentation on MENU.COM and mouse menus. Tablet drivers for mouse emulation in DOS and Windows.
Refers to reduced expectations on handwriting recognition accuracy, not to read human-illegible handwriting. Reference to Doonesbury commentary, use of point-and-tap selections and digital ink. Shows modified Graffiti character set (neography), Lexicus Cursive Recognition
Review of products and the problems of collaborative on-line editing tools for editors and writers. Suggests sticking with "traditional" mark-up symbols: easily learned, familiar, well developed for the problem domain. States pop-up or embedded comments awkward. Change tracking does not do well showing extensive formatting changes to users. Recommends digital ink (electronic ink) with gesture commands (the mark-up symbols), but commands not executed until approved by author. PenEdit (Advanced Pen Technologies), MATE, Red Pencil.
Proposal to use a DataGlyph (two-dimensional barcode) to having indexable/scannable information at the start of published articles. Compare to Silverbrook patents of 2004, and to Anoto. Note: there was a similar technology for a 2D barcode date for articles already in place in the 1980's?
Creating of a "handwriting-looking" font from samples of someone's handwriting: refers in claims to automatic classification/categorization based on slope and other characteristics of the characters
Travelling software program: interpretive code (P-code: compare mobile Java objects) moves from user to user to handle PKI cryptographyic signing and verification. Includes code-signing of program segments, transmit the certificate segment ahead of the program.
Signed authorization tokens (program authorization information) stating what capabilities (rights/permissions) a program may exersise. Tokens may be co-signed by multiple permission-granting authorities. Include checking timestamp and signature of program getting permission. Compare with DRM tokens?
Magic Lens project: application to database systems -- e.g. user requests map with information (from database) as to properties to rent. Magic Lens "filter" in window of GUI tool to change how information is display / what information is displayed.
GraspDraw project: Physical devices coupled to "virtual" objects via position sensors: blocks (bricks) that may be moved on a position-sensing surface. Refers to multiple-value input being either time-multiplexed (one mouse or stylus) or space-multiplexed (two hands).
Overview of problems to be addressed in mobile computing (e.g. tablets, smartphone, PDAs). Communications w. intermittent connectivity, file recognilization/updates, changing IP address. Small display, Presumes switch to touch/pen computing: touchscreen input less accurate than mouse, display obscured by hand/stylus. Power management (clock rate). EO/GO/PenPoint battery to 6 hours (same as laptop), Newton and Zoomer to 40/100 hours life.
Transparent windows in a GUI, using alpha blending of pixel data. Example show menu boxes with buttons overlaid on other windows.
Short study of Windows for Pen Computing handwriting recognition accuracy, compares to Neisser60, cites also LaLomia94
IDE development environment with object, code browser, links reference to source code. No layout editor for GUI shown.
Delayed translation/Deferred recognition of electronic ink, part of Slate/Bricklin thinking of ink as a data type. Implemented for PenPoint and Pen Windows in PenApps, a Visual-Basic like language. Refers to leaving electronic ink displayed if handwriting not recognized, or handwriting recognition result not found in dictionary. User can also display ink instead of recognized text. Gestures?
System administration manual for Unix. Discusses ACLs access control lists on AIX and HP-UX versions of Unix, with extended permissions, deny as well as grant permissions/rights.
Machine learning / backpropogation neural net work recognizing handwritten characters and symbols (gestures): training-based recognizer. Multi-stage recognizer: "energy" dynamic features (velocity and accelleration), and 3x3 grid patterns.
Guide to PGP public/private key cryptography and PKI
Collection of reference manuals and papers for Garnet IDE (see other refs) from CMU archive web site as a single publication/reference: includes directory listings.
Critique of Pen Windows / W4PC as only encompassing double-click, not drawing of gestures. Gedrics: gesture-driven icon. Gestures (of single stroke, to be fast) are performed on an icon: tab to activate, right-left to open, single-stroke question mark for help menu, closed "X" to delete, press/hold or hold/drag for copy/paste/move-to-parent, etc. circle left and circle right. Implementation on Newton. Does not mention "marking menus".
Video presentation of General magic smart phone / PDA. Slate computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Calendaring, virtual keyboard, electronic ink for notetaking, Personal Link communication network services from AT&T for email, etc.
Video presentation of General magic smart phone / PDA. Slate computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Focus is on Telescript: early messaging/email service?
Telescript: Prototype-based programming language for pen-centric pen computing applications for General magic platform.
GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Includes discussion of electronic ink, handwriting recognition, API for gesture recognition.
GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Class heirarchy and messages.
Visual programming language / GUI for creating programmed multimedia presentations
Capacitive/electrostatic grid tablet ("dipole electrodes"), finger affects capacitance at intersections. Coarse position is the grid crossing point with the highest value for change of capacitance. Interpolates between two adjacent grid crossing points for higher fine resolution to determine centroid of finger position. Proximity sensing. Says Rympalski measures changes in capacitance as conductive object comes near. Compare with multi-touch?
Speculative (forward-looking) paper on web-browser for PDA Apple Newton: advantage is ubiquity and portability, disadvantage was high wireless data charges while mobile, small display (four-way scrolling particularly difficult) and small memory. Deferred fetching of images (low bandwidth through workstation/PC to dial-up Internet connection). No bibliography: does not mention Lynx browser
Textbook/reference on 3D graphics algorithms: mostly curved surfaces, hidden surface removal, with emphasis on real-time performance (so nothing on ray tracing or luminosity). No masking.
3D/6DOF user interface for surgeons viewing CAT and other scans during surgery: Polhemus 3D digitizer inside doll's head, surgeon moves doll's head to show projected 3D image. Ignores X/Y position (so really 4DOF?), Z is used only to zoom image.
Capacitive grid digitizer, transparent on top of display
Extensive list of video tapes from ACM CHI, CSCW, and ACM SIGGRAPH Vieo Review relating to handwriting recognition, sketch input, signature verification, and pen-computing: 1983 to 1994.
1963 Video: Videotape of Sketchpad system by Sutherland
1985 Video: Software Control at the Stroke of a Pen - Pencept - CHI '85
1989 Video: The Paper-Like Interface, IBM Watson Research Center - CHI '89
1989 Video: Freestyle - Wang Labs - CHI '89
1992 Video: Combining Gesture and Direct Manipulation - CMU
1992 Video: Touchscreen Toggle Design - Univ. of Maryland
1992 Video: Tactile Manipulation on a Digital desk - Rank Xerox EuroPARC
Video: Alternate Reality Kit - Xerox PARC
Video: The Lapidary Graphical User Interface Design Tool - Univ. of Tennessee
Apple Newton: selected text in text editor can be copied/pasted by drag-and-drop: shaded image of text is what user drags around. Compare with floating icon for drag-and-drop in PenPoint?
Overview of PEM Privacy Enhanced Email PEM standard: certificate based encrypted and originator-authenticated/signed email. File also contains RFCs.
RFC 1421: Privacy Enhancement for Internet Electronic Mail: Part I: Message Encryption and Authentication Procedures (1993)
RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management (1993)
RFC 1423: Privacy Enhancement for Internet Electronic Mail: Part III: Algorithms, Modes, and Identifiers (1993)
RFC 1424: Privacy Enhancement for Internet Electronic Mail: Part IV: Key Certification and Related Services (1993)
Touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials.
Functional touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. (Screen is square, rotation is immediate.) Pop-up tools have window frame and draggable title bar. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials.
Touchscreen PDA with full keyboard, swivel/convertible/clamshell display, penGEOS software. Front cover shows OmniGo 100 with both portrait and landscape screen orientation. Trademark on "hotsync" "hot-sync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable). GeoWorks Bindery for electronic books with hyperlinks in text. IZL (Interpreted Zoomer Language) programming environment for GeoWorks. Septor wireless pager/news-alerts services .
Windows software and serial cable for syncing and transfering between OmniGo pen PDA and a Windows PC. Trademark on "hotsync" "hot-sync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable).
Pen GEOS operating system documentation.
Looking for optimal trade-off of transparency between user attention being focused on one of two images, versus divided attention between the two (Stroop Test). Show semi-transparent windows, semi-transparent pull-down menus, pop-up menus over graphics and over text. Also shows 100% transparent, with only menu labels / lines visible (in contrasting puke-green color) with shadowed outline for contrast. See Rockstar Samsung.
See-through/transparent image over background image gives illusion of depth (depth multiplexing): superimposed windows, overlapping windows. Also changing physical depth of images (semi-silvered mirrors) in 3D. See Rockstar Samsung.
Show semi-transparent pop-up menus (including pie menus or context menus) semi-transparent or wire-frame over object, so that user can still see context, underlying image/object not obscured. See-through UI tools. See Rockstar Samsung.
User interface with semi-transparent windows, menus, dialogue boxes, other objects with attention to "layered" interface objects, click-through tools. Cites to Knowlton on dynamically re-labelled buttons and keys. See Rockstar Samsung.
Commercial product that finds literal names based on capitalization, and dictionary, plus knowledge engineering rules. Compare with regex expressions to find phone numbers. Finding names, combine with UI?
Xerox data glyphs, 2-D encoding of information in small pattern within printed images. Diagonal pattern segments, 0 and 1 representations at right angles, but small enough that do not appreciably affect visual appearance. Made robust to rotation by encoding reserved pattern segments that are used to recognized the orientation.
Gesture (?) for turning a page by touching edge of page in a simulated book, and moving animated page(s) to the left or right. Compare with flick gesture, PenPoint OS metaphor of notebook pages?
Translucent/semi-transparent window on OS/2 so as not to obscure image underneath (touchscreen/mouse). Blending for transparency is done by checkerboard of BITBLT, so that background image with colors appears visible.
Hybrid user interface: touchscreen UI can be combined with 3-D input device, because user can reach out to touch touchscreen with the 3-D input device in their hand. Relative vs. Absolute gestures: hard to perform gestures in absolute coordinates. Bi-manual/two-handed input advantageous over sequential single-hand input, because user can (e.g.) position a tool with one hand and use the tool with the other. Mentions using multiple pointing devices in a prototype (trackball and mouse, or two mice). User's spontaneous use two hands for combining translation/rotation/scaling gesture (direct manipulation). Compare with multi-touch.
Scroll to line, paragraph, or other document boundary. Scrolling initiated by minimum gesture movement, with second threshold to stop autoscrolling or scroll back if touch location has not moved on touch-sensitive surface.
Touchpad controller for very small touchpad: uses (optical) sensing of velocity to position cursor faster if touch is moving fast.
Introduction to Smalltalk language and environment. (Electronic file.)
Invariant handwriting features (independent of rotation, translation, scale) are ratio of tangents and normalized curvature, for Hidden Markov Model recognizer
Stochastic pattern recognizer, Hidden Markov Model (HMM)
URL as of 5/9/2002: http://www.research.avayalabs.com/user/jianhu/index.html with additional information on Jianying Hu.
States that using a sliding window in a recognizer reduces error rate by 50%, but does not give the point of comparison, or the description of how the error was computed (false positive, false negative, etc.)
Resistive-film capacitive touchscreen with front shield to protect from stray capacitance
Acoustic wave touchscreen: hard stylus presses on top elastomeric layer, which then absorbs acoustic energy of wave propagating in layer underneath
indexing of electronic ink notes by matching scribble markings: states 97% successfully matching rate (compare with character recognition rates of 97%), Scribbles/blobs taken from fields: e.g. "To:" fields in notes/email. Feature points are knots/inflections and syntactic rules for sequences, compare with chain codes.
Architecture of IBM Pen for OS/@ V1.0 and PenDOS from IBM, version 2.2: Lists gesture assignments by Session Type, Gesture Commands and Sessions
IBM Simon user manual. Touch screen uses plastic stylus or finger. PredictaKey keyboard shows six most likely characters on virtual keyboard, Other shows next six most likely. User can highlight numbers in messages and they are dialed from a pop-up menu. Physical slide switch p. 4 for On/Off / Resume/Suspend, p. 8 Simon is not completely off. If not off, touching screen turns unit on. Buttons with broken-line edge are not available: drag stylus across screen to highlight button, then lift stylus to select that feature. Some buttons display check mark on/off on touch. Folder-shaped buttons contain additional buttons. Has search function for address book p 35. Specifically states that operations happen "when you release the button" e.g. pen/stylus lift, not on pen/stylus down -- carry-over from mouse development?
Windowing system uses different control layout from Microsoft Windows. Drag-and-drop to printer icons. Style manager allows color palette for all windows to be updated. Three button mouse, pop-up menu button can be rightmost of leftmost button depending on handedness of user.
Unnarrated demonstration of IBM Simon from BellSouth touchscreen smartphone, circa 1994
Demonstration of IBM Simon touchscreen smartphone, circa 1994, from reporting on CHI 2011 conference.
Pen/OS2 pen computer: every pen touch-down event on an object, object determines how stroke should be interpreted: e.g. mouse-mode in title bar, mouse+gesture mode on button, mouse+gesture+single-character-recognition in list box, etc. User can override definitions manually, or by holding stylus side-button. Compare with "gesture buttons", ink interpretation in PP and PW?
Readme file (and file listing) of installation media for Pen extensions to OS-2 operating system, 1995. "Enable Edit Window" -- first point of contact (in one edit window) causes all other input to be directed to that window regardless of where. "Telepen" whiteboard application? Some incompatibilities with pen as mouse.
Computer hardware technical dictionary
Overview of problems in handwriting recognition, both OCR and on-line. Includes discussion of loop/cusp transition (i discussion on segmentation of script characters)
CodeWarrior by Metroworks, Object Master, MacAPP, Workshop. Development environment: gives 1994 or earlier date for MacAPP and CodeWarrior
Microsoft WinPad palm-top/hand-held pen computer, high price and release delays resulting in Compaq not releasing Mobile Companions. (WinPad later named Microsoft At Work for Handhelds.)
Appears to have no relation to Inforite82 product, separate company? Several companies have used the name "Inforite"?
Intel386 embedded processor, includes registers and MTL6560 digitizer controller for resistive touchscreen (digital voltage measurement), controller/co-processor for LCD
resistive touchpad using two sheets separated by an air gap, four-wire sensor. Based on contact resistance, also determines touch pressure (force). Tap-and-drag gestures. Drag Edge lock: if drag to edge, maintains drag gesture status for three seconds after release so that drag can be continued/extended with an additional drag/wipe gesture. Further gestures include right-click by tapping in upper right corner region of touchpad digitizer, right edged border scrolling for up/down scrolling.
States large hardware business for specialize electronic chips for PDAs with electronic ink, handwriting recognition coprocessor. Mentions immaturity (performance defects) of digitizer tablets. e-ink.
User interface, set windows to transparent so that user can see the information they have selected.
Handwriting recognition UI characters translated to keyboard input: no references to Sklarew/Linus or Ward/Pencept?
TeamWorkStation-1/ClearBoard: Whiteboard systems: Combined videoconference image in shared drawing space on workstation display. Using transparent display, other user appears in video on through transparent display, writing reversed. Cites to Media Space shared drawing space from 1988. "Transparent digitizer sheets" -- does not say what touchscreen technology used.
Describes mouse as absolute position device if not lifted from pad (sic), thus a three-dimensional digitizer is equivalent for two dimensions. Refers to device-independent API interfaces such as ACM CGS. But in practice, practical attributes of input devices are not separable from types of user interfaces and interactions.
Wearable computer, a PC (with standard PCMCIA cards, etc.) with components distributed in pockets in a "utility belt", cables/connectors for system bus etc. running through belt (or suspenders). Refers to foldable/portable input/output device for pen-based LCD (or keyboard/touchscreen).
LabView graphical programming language for software simulation and control of virtual laboratory instruments, describing dataflow programming. Text refers to RUN switch in examples (p84/92). Introduction gives history of LabVIEW to 1986. User interface: notes that scroll bar on MacIntosh and Windows effectively halts asynchronous/multi-tasking operation.
Toolglass two-handed/bimanual user interface: non-dominant hand positions an object or tool, dominant hand uses or manipulates the object or tool.
Survey of mathematical expression editing, mostly from a keyboard (and mouse?). Section 4.10 talks about pen computing user interfaces: two-dimensional input is good, but large number of symbols used is problematic for handwriting recognition
User interface with touchscreen, electronic ink, gesture commands, and handwriting recognitions. System determines how to interpret user input based on context such as size relative to known characters, location of input on/over/awa-from existing ink or characters, whether the input document already exists, and relative priority of gestures.
Additional materials in file: winster.com, etc. historial notes on PenPoint, concerning Jerry Kaplan
History of GO Corporation and PenPoint, experience with IBM and Microsoft, from point of view of CEO of GO. Compare to account of same events by Marlin Eller in "Barbarians led by Gates" about PenWindows.
Additional materials in file: winster.com, etc. historial notes on PenPoint
Study of testing handwriting recognition algorithms using some 87,000 characters from 150 writers. Supported by Microsoft Pen Computing Group: Greg Slyngstad and Sunh Rhee. (Cites variability models of Ward and Kuklinski, but does not derive implications about 97,000 being a small sample size.)
Haptic mouse with force feedback, integrated with X windows. Electromagnetic actuator in mouse, feedback provided external to host CPU because control loop through host application too slow or too low in bandwidth. Turns mouse into absolute position device?
Proposed API for handwriting recognition systems, using pen strokes optimized for point vectors in the X window system, but otherwise independent of OS or windowing system
Incremental translation/recognition of handwriting
support for context as for Japanese and Cursive English
API for character prototype files and dictionaries
Data structures for gestures and arbitrary objects
Data structures correlations between recognized objects and pen strokes
Handwriting recognition system for pen-based computing for Solaris: cited to PenWindows/Microsoft Windows for Pen Computing (MWPC), PenPoint, Apple Newton for pen-computing. Solaris API based on PenWindows API and Penpoint. Not integrated tightly with windowing system: multiple window toolkits on Solaris (XView, OLIT, Motif, COSE, Interviews). Stateful implementation: can acquire context information from previous strokes. Recognition subsets of gestures, mathset / math symbols, Kanji (JIS1 subset only), Katakana, hiragana, etc. Support for delayed strokes vi correlation.
Averaging out the prototypes in adaptive handwriting recognition system
Transparent overlay windows in a user interface
Third volume in "Graphic Gems", collection of papers: Image Processing (bitmap compositing), Numerical and Programming Techniques, Modeling and Transformations, 2D Geometry and Algorithms, 3D Geometry and Algorithms, Ray Tracing and Radiosity, Rendering.
On-line help, with hypertext linking and search for related topics by text
Capacitive grid/matrix of conductors used for touchpad sensor for fingerprint scanning. Compare to grid arrangement of Rekimoto.
OS for hand-held personal information manager: Telescript language for application implementation. Touch-sensitive touchscreen, stylus or finger. Refers to targetting "halo" for objects. (Compare with Slate PenApps "gravity"). Tap/Touch gesture terms used interchangeably. Desktop/Office/filing-cabinet/hallway metaphor. Electronic ink note-taking. On-screen keyboards: brings up "smart guess" keyboard (numeric, etc.) based on syntax of input.
Simulation of indexed flip book / rolodex in Wang Freestyle, with small edges of pages shown.
Feature in LabView visual programming language for virtual laboratory instruments. Fig. 5 , text page 70 refers to slide switch: compare to Apple slide-to-unlock?
History and technical description of Kerberos authentication system through version 5, at M.I.T. Project Athena. Single sign-on: user login gives ticket used by all application. Uses secret-key (single key) cryptography, version 5 addressing interoperability with public-key cryptography: cites to RSA digital signatures. See also earlier 1991 publication.
Vibration haptic feedback on remote-manipulation display, combined with/without force feedback. Illustration is two finger "waldos" to prevent user pressing too hard.
Overlapping translucent windows (or non-rectangular patches), so that users can annotate on top of other windows. Implementations: re-drawing (non-pixmap?) on top of other drawings after lightening lower window, changing color of lower pixels by translucency of pixel above (alpha blending). Interesting approach on sketching, electronic ink. Patch-related drawing gestures: line into (pearl), zig-zag clear (erase), zig-zag over border (cross-out/delete), small circle for pop-up, clockwise/ccw spirals. Gestures vs. ink by side-button on stylus/pen tablet (compare Pencept recognition macros). Overlapping windows may cause lower patches to render differently (e.g. not show control elements, etc.) (cites Toolglass). See IntVen Motorola, Rockstar Samsung.
User interface to select objects using selection rectangle (objects may intersect edges), the selected a tool: Scale, Skew, Rotate, etc. Moving rotate tool changes/sets pivot/rotation point.
Dual-display device, two displays can be used as one (two pages of a book), or used separately. Compare with devices like Refalo? Uses term "electronic pen" for touchscreen. May be used as add-on display to desktop computer. Thumbnails of document pages on one display, full page on other. Can display virtual keyboard on one display or portion.
Use of pop-up radial menus to train users to "gestures" which match the motion used to select from a radial menu. Also display of images of gestures in menus, and pop-up text giving name/meaning of gesture recognized. Circle/lasso gesture around an object brings up a pop-up menu of actions that may be applied to the object. Also shows pig-tail gesture for delete/erase, scroll-up and scroll-down (flick?), lasso gesture for select. Gestures may be recognized using velocity: e.g. Slow flick moves to next page, fast flick to end of document. press-and-hold/"pressing and waiting"
Marking menus on MacIntosh/Apple. (pop-up pie menus when user hold stylus still, selectable without first lifting pen). Zig-zag gestures for navigation. press-and-hold / press-and-wait for 0.3 second.
Pressure sensitive tablet digitizer using two conductive sheets and conductive rubber
Cited along with Neisser (93% recognition success for humans without context), that users not satisfied with 97% correct character-to-character handwriting recognition success.
pop-up virtual keyboard in GridPad computer, in a particular form application. Compare to Soft input panel SIP?
Basically a PDA/smartphone/always-with-you-computer with notes, reminders, calendar, video diary that is also sensitive to user's context (especially location). Intended to help look up information, or recall information for user. Cites to Memex as collection of user's notes and documents. Memory aids: e.g. easier to remember who was at meeting if other aspects of meeting are recreated / reminded. Cites to Amstrad PenPad PDA, EO 440 Penpoint, Newton Messagepad, etc. Main example is Xerox PARCtab, touchscreen, wireless/infrared, active badge (fine-grained location).
Front-mounted electromagnetic digitizer (CalComp) with a display: gives long description of Scriptel front-mounted electrostatic display, discussion of problems of rear mounting. Uses term "pen-driven computing". Two layers of grid are fabricated on two layers of transparent mylar.
Forms-based inputting gestures to control recognition, insert, delete scroll -- compare with GO
Refers to smoothing (and averaging?) to obtain the non-distortion point on stylus input filtering?
On-screen toolbar for selection one of several on-screen virtual keyboards or handwriting input, toolbar can be disabled: handwriting recognition shows multiple candidate results, user picks correct one. Toolbar can be repositioned on screen: compare with Pen Windows and PenPoint?
history of handwriting recognition (?) back to 1950's.
Study of digraph/trigraph dictionaries in virtual keyboard typing correction: a single trigraph dictioary not as useful as multiple digraph/trigraph dictionaries with different word populations / vocabulary for appropriate domains.
Lexicus neural network handwriting cursive recognition: 25,000-word dictionary and statistical methods (compare with Viterbi algorithm?)
When zooming on graphical image/map, show birds-eye or world or greater context view as translucent overlay on window, so that user can maintain visual context. Zooming on file system heirarchy zooms into files, without explicit opening of file. Cites to Bier for "see-through" interface (transparent pixels instead of translucent overlay).
Microtouch touchpad digitizer: press-and-drag distance is a minimum distance threshold before user input does drag automatically by simulating mouse button action. Dwell/press-and-hold for minimum distance threshold.
GPS aid for visual impaired/blind, giving information about surrounding environment as well as location. Output to user by means of artificial auditory beacons located at correct/appropriate binaural location relative to user.
Avoid need for handwriting recognition when indexing documents by approximate matching of hand-drawn pictograms. Mentions difficulty of handwriting recognition of (Asian) languages. Compare with Slate DayTimer/PenScheduler for ink indexing, Windows for Pens NoteTaker.
Searchability of electronic ink; approximate matching of hand-drawn pictograms, not just comparing with character recognition results to index handwritten text
Review of pen-based computer (PenPoint) with applications for engineering: notetaking, engineering design. Applications written in Slate PenApps. Generally, paper fared better than electronic notebook, in part because notebook was not perfect writing surface and was a bit small (6" by 9"). Notes that engineers dislike doing documentation, in part because words are not the natural medium of expression -- diagrams/sketches are preferred. Cites several other electronic/virtual notebook systems, but most not portable.
Project planning/project management application. Tasks, Milestones, assignment of resources and personnel, project tracking. Supports critical path method CPM. Views include PERT charts, Gantt charts, Detail charts, Calendar views.
Computational rendering and display of holographic (stereo vision: full 3D possible) images in "patient" real-time (2 seconds). Can also render a real-world scene captured from a linear array of cameras.
Pen computer can be rotated so that screen orientation is portrait or landscape, for right-handed or left-handed user. Mechanical button to select orientation.
Determine user intent by applying a pattern template to user interactions in a pen user interface, and then carry out an operation automatically.
Also check Capps file
Speed of entering text on pen-based (PDA/tablet) computer: on-screen/soft keyboard with QWERTY basest, then hand-printing recognition, on-screen/soft keyboard in two-row Alphabetic arrangement slowest.
Application of Fitts' law to 2D and 3D user interfaces.
Product announcements on Tarpon, ruggedized Newton MessagePad with wireless modem (Ardis network), GPS, cellular phone ("external industrial phone receiver").
Description of Wellner-DigitalDesk-like interactive project for capturing handwritten (paper) updates to CAD drawing. Overhead Projection of virtual paper onto physical paper, location of physical paper tracked optically. Live video communication between users.
Shows pen/stylus based PDA with office desk / office room and hallway metaphor. Appears to be a resistive touchscreen, looks like it might be a Data Rover 840.
Dynamic time warping / Time Delay Neural Network for single-character recognition, using a 3x3 or 20x30 bitmap local image
Claims 99.5% recognition writer-dependent, between 91.5% and 99.5% on writer-indepedant: expansion to larger writer-independent database gives about 75%
GUI frameworks comparisons: Macintosh Desktop, OSF/Motif Desktop (with three-button mouse operations BSelect, BTransfer, BMenu right button), NeXTSTEP Desktop, IBM CUA Workplace Shell, Microsoft Windows Desktop Environment. Does not cover PenPoint, page 191-ff covers Microsoft Pen Windows extensions: press-hold-drag is a selection operation, PenWindows 1.0 gestures Extende Backspace, etc. Direct manipulation. Pop-up menus on page 34 and 202. Describes drag-and-drop and pop-up menus as OOUI, object-oriented user interface, not GUI.
Projection whiteboard system with LCD project panel, overhead projector, digitizer. See other references to MartinD.
Describes alignment/calibration method for display and projection vs. tablet, similar to that of PenCept/CIC/etc. for paper versus tablet.
Knight-Ridder Information Design Lab: electronic newspaper on a pen computer, with pen/touch user interface
Bibliography of papers relating to document understanding, starting about 1980. Sections include: On-line character recognition; Gestures and Sketches; Script; Foreign optical character recognition; Recognition of mathematics and Formulas; Line drawings; Signature verification; Writer Identification; Synthetic data; Forms processing; Shorthand; Music recognition
scrollbar slider: user can control speed of scrolling (for finger control) based on pulling a virtual elastic line farther or less (or changing direction of pull). Cites to AlphaSlider. Note: cited in at least one scroll/bounce patent by Apple.
Chapter 5: Interactive learning / Concept learning system Cima, Turvy, and Moctec. Example is teaching system to find phone numbers in notes, in different contexts and in various canonical forms, and second component that learn so interact with users and application programs, (dialing numbers)
Slide an icon onto a position to unlock: slide to unlock?
Pie-pad pop-up menu for numeric input, pop-up circular menu with digits. Marking menus. Mentions Goldberg 93 paper on unistroke.
Includes reviews of several experimental systems of the early 1990's, and a lengthy bibliography: several German references included. Almost all the references are from 1991 to 1995, very little before that.
Unistrokes. Jotted notes: implying electronic ink as a data type. PenPoint, Windows for Pen Computing. Cites Alan Kay for early vision of handwriting computer (although no such device was ever built?). Cites early paper by Ward, but not the on-line bibliography (1995). Rotatable orientation of pen computers: landscape/portrait.
touchscreen/tablet/digitizer: mentions dual sensing of stylus/pen or finger or both, with positions of each.
Tap and hold (TapNHold) gesture: tap analogous to clicking left mouse button: TapNHold click tip of pen on tablet and holding for a specified amount of time. German edition of (discontinued) support article documents Windows for Pen Computing 1.0, prior to Windows95. Tap/press-and-hold on Pen Windows
Windows for Pen Computing as portable information at your fingertips: reference "vision" video of Information at your Fingertip in 2005, with face-to-face video in mobile settings, pen computing, video conferencing.
Windows for Pen Computing version 1.0 user manual. Pen tools group includes Pen Pallete (soft input panel for writing/keyboard), on-screen virtual keyboard. Sample form has boxed handwriting input, unboxed input with timeout for word segmentation.. Checkmark gesture to edit text (similar to PenPoint), pigtail gesture to delete, space-insert gesture with hotpoint. Tap and hold gesture (press-and-hold) to start selection stroke. Newline, copy (sideways pigtail), paste (inverted V), delete, delete word (right-left-combined), extend, tab gestures.
Cursor automatically turns to pen for handwriting input in application when cursor is in a text-input field. User-defined gestures are circled letter gestures. Trainable recognition (note that Toshiba unit ships with CIC recognition). Control panel settings for double-tap gesture time and area settings, press-and-hold delay. Press-and-hold is effectively a separate gesture recognized by distance threshold and time threshold: moving before time threshold starts a gesture or ink (with recognition).
Handedness setting changes location of menus. Control panel rotate setting to switch orientation of display landscape/portrait. Notebook application: barrel button on pen to display ink eraser; deferred recognition: select ink (by tapping) then select text tool. Recognized text optionally sized according to ink dimensions. Shape recognition (circles, squares). Equations (= at end) automatically calculated; outline indexing of notes. Chart of recommended printing styles.
List of third-party accessibility extensions for Windows and Windows NT. Screen reader / screen review, screen enlarger, on-screen keyboard / virtual keyboard. Vendors include Magic Deluxe / Microsystems Software, MAGNUM Deluxe / Arctic Technologies, MagniPORT / Microsystems Software, Number Nine (hardware pan and zoom in display card), Vista / Telesensory Inc., Zoom Text / Ai Squared, JAWS, outSPOKEN . On-screen keyboards include word prediction, click or dwell mode / AutoClick. Screen Keyboard (SKB) in Windows for Pen Computing.
Microsoft Pre-release feature description for Chicago / Windows95. Plug-and-play, Toolbar (for cut-and-paste, etc.) in MS-DOS command window. Support for multiple pointing devices (mostly for accessibility: alternative head mouse, etc.). Nothing on SIP Soft Input Panel, PenWindows or tablets.
Windows UI guidelines include keyboard, mouse, and Windows for Pen Computing / PenWindows gestures: General circle-around-UC-letter gestures, has circle-M/M-circle pop-up "context menu gesture" / "pen menu gesture". Fundamental behaviors defined for a Pen are Pressing/Tapping/Double-tapping/Dragging. Gestures leave ink. Lasso-tap (circle-tap) selection gesture. Tap on text box brings up text-input handwriting window, with alternative word list or alternative character list from recognizer.
Partial print copy in physical file -- see electronic copy at http://www.ics.uci.edu/~kobsa/courses/ICS104/course-notes/Microsoft_WindowsGuidelines.pdf (fetched 2010)
stylus or finger, pen can perform equivalent operations of mouse.
Defect in tablet/pen driver in Windows for Pen Computing: circle-m context menu gesture does not emulate right mouse button, other problems with barrel button on stylus
Similar topic to Microsoft95a -- includes section on special design considerations of Accessibility. Some sections on pen input. Guidelines on targeting of electronic ink to fields (but not built-in? Compare with Slate PenApps "gravity"). Lasso-tap gesture to select. Need larger tapping (double-tapping) target with stylus than with mouse, similar considerations for targeting of lasso-tap selection gesture on text. Pen menu (context menu) gesture is circle-m. Describes activate-on-lift of mouse being applied for pen: "Do not activate the window unless the user releases the mouse button (the pen tip is lifted) in that window." Press-hold-drag for select instead of barrel button. Auto-scrolling based on velocity. Mentions accessibility "Screen Review Utilities" (screen readers) for the bling, on-screen keyboards. Pop-up/context menus synonymous: refers to contextual interfaces such as pop-up menus, property sheets, and OLE drag and drop. pop-up (context) menu handlers. Press-and-hold for select, p. 55.
Partial print copy in physical file -- electronic image file.
#include definitions file for Windows for Pen Computing Version 1.0 -- 1992? Pen tablet data OEMPENINFO include pressure (force), height (proximity), angle X/Y and Z, barrel rotation. Standard gestures for clear, extendSelection, undo, copy, cut, paste, clearword, kkconfer; AnsiGestures (ASCII?) backspace, tab, return, space; application gestures circle a-Z and circle A-Z; shapes line, ellipse, rectangle, min and max
See also penwin.h for version 2.0
Accessibility on-screen visual/virtual keyboard systems for accessibility: compares to single-switch Morse code. Eek!; Microssytems Software HandiKey; My-T-Mouse; SAW Switch Access to Windows; SoftType; WiViK2 Windows Visual Keyboard;
Features include word prediction, dwell-mode pointing, access grids, mouse emulators (head-mouse, etc.), keyboard macros
See also similar Microsoft technical notes from other years
Device to convert absolute co-ordinates of touchscreen to mouse relative-motion coordinates, and having "buttons" recognized on the touchpad for the mouse button: prior art include MicroTouch Unmouse cited 1990
Compare to Schumer patent on tablet controller, where driver functions are claimed if located in an external controller for a tablet (or mouse?)
Sketch/diagram editor employing constraints, and dealing with inconsistent state during direct manipulation editing: flowchart, hypergraphs
Force-feedback joystick, with multiple force actuators. Software to create feel of textures, virtual spring and inverse spring forces, etc. Includes reference manual for Sandpaper software.
When selecting objects by selecting a range (drag selection rectangle?), take into account time ordering of when objects were created: user is likely to want to select objects created by the user at about the same time. Uses a white board with rear projection.
Whiteboard system (Xerox Live-Board Tivoli / MeetingBoard on Windows PC) with free-form interaction -- primarily electronic ink scribbling: recognized rough layout of lists, text, tables, and indented outlines in electronic ink, based on informal structure or "visual grammars". Mentions aha! InkWriter which formats/indexes ink based on handwriting recognition, but does not compare with van Raamsdonk which edits directly electronic ink. Selection via various bracket and corner gestures, insert via caret gesture. Has "table clean-up" function to align slanted writing lines of electronic ink.
Port to Windows of Unix Programmer's Workbench: compare with cygwin
Port to Windows of Unix Programmer's Workbench: compare with cygwin
Simplified unistroke-like gestures, as substitute for writing full characters. User writes on a 12-segment outline of characters, outline in shape of figure eight, similar to seven-segment display. Punctuation and some other characters input via pop-up keyboard. For Apple Newton. (See also AlphaPad00)
Radius Inc. Display: rotating display between landscape and portrait orientation results in all windows being rotated, windows may also automatically layout GUI elements.
Motif Window Manager: 1994 version, updated as of 2001. resource XmNsnapBackMultiple sets snap-back/bounce distance if use moves mouse cursor too far outside of bounds of scrollbar widget. Automatic Layout of GUI elements? XmNselectThrehold: click and drag minimum distance that pointer must move to select text on a drag.
Motorola Envoy communicator: PDA with General Magic Magic Cap, stylus/finger touchscreen, fax, wireless (ARDIS cellular radio) /wired modem. PenWare PenCell spreadsheet, AOL-online, Pocket Quicken, online maps, terminal emulation (e.g. vt100), on-screen virtual keyboard, electronic ink. No voice phone.
Video demonstration of IBM Simon touchscreen smartphone: games, apps, scrabble, text messages, data plan
in Garnet system papers: Prototyping user interfaces can be difficult, because test-subject users may not behave like real users, may even have a hidden agenda. Coupling of user-interface to application (via application call-backs) is a nightmare in maintainability. Shows dynamic menus.
Garnet system: Constraint toolkit for UIMS, including object management in user interface
Garnet system: Constraint toolkit for UIMS, including object management in user interface
Garnet system: Direct manipulation user interface and IDE with Interactor objects for mouse, rotating, text editing, gesture recognition. Include Lapidary to create widgets or objects, Marquise tool to demonstrate/animate user interface behavior of applications.
Copy of presentation video in electronic files.
Garnet system: supersedes CMU-CS-92-114. States gesture UIs used since 1970's in commercial CAD tools (probably Applicon)
Overview to Garnet System 3.0, gesture-based/direct-manipulation user-interface project. Introduction to full set of Garnet documentation, "The Garnet Reference Manual: Revised for Version 3.0". Extensive bibliography. Garnet: "Generating An Amalgam of Real-time, Novel Editors and Toolkits". IDE which allowed developers to "draw" the user interface using graphical objects: user-interface styles could be applied to all user-interface/objects (look-and-feel independent). Slots (methods): compare with PenBasic from Slate. "X" gesture to delete (using tablet?).
in Garnet system papers: describes X-Windows lack of agreement on widgets, therefore Xt intrisics so support. 3-D GUI toolkits: Brown Animation Generation System, Silicon Graphic's Inventor toolkit.
See also Myers 93 reference, same title.
FingerMouse: Hands-free/Glove-free pointing input: downward-looking cameras (stereo) recognizes pointing finger when held above keyboard. Shift key used for mouse click (one-button mouse). Concept originated with Francis K.H. Quek.
Practical guide to Security features and practicies in POSIX (Unix): MAC Mandatory Access Control, DAC Discretionary Access Control, ACLs, Information labeling. Public key cryptography PKI. SQL and network communications security (e.g. weakness of TFTP trivial file transfer protocol), X.400/X.500 standards, APIs for certificate storage and access (compare CryptoAPI, CryptoKI), certificate revocation lists. Mentions ECMA standard 138, "Security in Open System" with attribut certificates. No list of references? (e.g. Multics)
Optical pen stylus digitizer that takes CCD images of physical ink trail of pen near tip, whole image and writing dynamics can be reconstructed from the sequences of images. 80% (40/50) success rate on reconstructing whole images. Difficulty reconstructing long straight lines. Stress sensor (force sensor) for detecting when writing. Sampling/data rate 10 snapshots/second. Compare with Anoto.
Transparent X/Y digitizer/touchscreen using transparent electrode wires of ITO indium tin oxide. Stylus/pen uses a plastic film coating to get appropriate writing friction on a metal core stylus.
LabVIEW Graphical programming language and software for Windows 3.1: Date approximate. Contains Wells book.
Part of NeXT Computer SDK. Overview of GUI tool for building/making applications. There may be an earlier version: this appears to be 3.X circa 1995. refers to bundles (which can be applications or collections of resources), and projects contained within other projects.
Part of NeXT Computer SDK. Overview of GUI tool for building GUI user interfaces. There may be an earlier version: this appears to be 3.X circa 1995. See also "Working with Interface Builder": build UI from palette of icons/views/components.
Tactile pressure/force sensing skin on cylindrical fingers, and spacing and depth of sensors to "feel" shape of objects with fingers: multi-touch. Strain values used to determine radius of curvature of indentations in elastomeric/rubber skin. Compare with M.I.T. "Teddy"? Review of capacitive, conductive rubber, piezoresistive, electrorheological, and ultrasonic skin sensors that can be constructed in arrays. Design useds capacitive sensors measuring changes in plate separation.
Electrostatic/mutual capacitance imaging proximity sensor for objects: single pair of electrodes to sense distance, pair of electrodes moved over object in a known plane (or otherwise)
History and screenshots fro WinPAD Beta, precursor to Windows CE. Stylus on handheld computer, electronic ink annotation. Manual change of orientation/rotation.
General information on PC DOS and PenDOS: mentions "mouse emulation" in PenDOS, separate from tablet/stylus driver: limited use of Gestures in pen-aware Mouse applications. (Web site partially in Polish)
Pen Computer for Windows for Pen Computing. Special VGA display drivers (for inking functionality?) for screen rotation at 0,90,180,270 degrees orientation.
Press report on trade-secret theft lawsuit between VMI and Autodesk, concerning Otto Berkes, later director of UMPC/Origami project at Microsoft
Review of Mosaic Input, Zone-based handwriting recognition using figure-eight overlay. Awkward to learn, vendor claims 40 words per minute input rate with practice.
Pen tablet PC using Wacom digitizer. Mentions "Dynapad" by Alan Kay: Mistaken reference to Dynabook? Provided with choice of PenPoint, PenDOS, Pen for OS/2 Warp, PenRight!, or Windows for Pen Computing 1.0. Uses ThinkWrite handwriting recognition instead of Microsoft MARS recognizer. Windows has option in Control Panel to set screen orientation, but off-screen controls more suitable to portrait mode orientation.
Casio PDA: does not mention pen or handwriting input. "handy terminal" vendor description. Comparison with Panasonic JT-900PT. Includes printer.
Windows for Pen Computing version 1.0 not easily upgradeable to Windows 95 -- now refereed to as pen service Windows for Pen. Windows for Pen Computing development environment: Visual Basic with version 2.0 pen services penctrl.vbx updated to pen2ctrl.vbx.
Mentions HEDIT and BEDIT (boxed edit) handwriting input controls shipped with Visual Basic. Windows for Pen Computing development environment: Visual Basic with version VB3.0/Pro with provided HEDIT and BEDIT controls.
PenRight by Philip Shoemaker: MSDOS, MSDOS DPMI, and MSWindows pen-enabled platforms. Includes cross-platform support for Graffiti. Windows for Pen Computing development environment: PenRight! for Windows with integrated graphical environment, update to earlier PenRight! Pro SDK. Cross-platform to GEOS (MSDOS based), Casio Z-7000 and Tandy Zoomer PDAs. States that Palm Computing has a handwriting recognizer for MS Windows (Graffiti), but not shipping.
HOPE! for porting applications from Visual Basic to a PDA that does not run Windows: emulated Windows user interface. Pen Computing development environment: supports Casio Z-7000 PDA and Casio PA100 pen computing products, which run GEOS OS.
Description of Newton OS NOS ("Dante") 2.0. Mentions selection and dragging of one icon, or of multiple icons. Feature also in Windows desktop?
Windows for Pen Computing with integrated wireless radio. Handwriting recognition by ART, or cursive recognition by Lexicus.
1994 Video: Demonstration of PenPoint / EO unit. YouTube comments mention direct manipulation and gesture UI, also demonstration of convertible tablet computer before Microsoft did a convertible. Shows landscape mode versus portrait orientation/rotation. aha! InkWriter shown for note-taking. MagicScript programming. First few seconds are part of a CIC PenDOS / Windows demonstration.
Windows for Pen Computing version 1.0 versus version 2.0, boxed edit control, hedit handwriting edits
Lisa Sampfli, PenFact Inc. in Boston Massachusetts
Gestures p. 61: Circle gestures with letter: Lasso: p. 481 shows list of circle gestures by shape, using mnemonic characters -- compare to "recognition macros" of PenCept; HPENDATA object holds chunks of electronic ink data, separate from mouse messages. WM_LBUTTONDOWN / IsPenEvent() / DoDefaultPenInput: electronic ink data is associated as extra data (HPENINFO) attached to a mouse event. Defines stroke pen data between two pen tip transitions, hence touching strokes and in-air/proximity strokes. (What about out of proximity?) Context menu in bedit (boxed handwriting edit) controls. Single tap and double tap on character bring up a (context) menu/list of alternative characters/words. Page 41 portrait/landscape rotation of orientation mode
Tactile output of GUI for the blind: two-line Braille Display can be moved around screen image in discrete steps (no mouse), image on screen translated to Ascii/Braille characters: brackets, etc. Routing button "drops" mouse cursor at current location of Braille display in window.
User-interface for large surfaces needs UI without static-placed element (e.g. pull-down menus). Pop-up pie menus, hierarchical menus, pop-up handwriting recognition pads, gestures, vocal user interface. Problems studied in example of Xerox Liveobard whiteboard system. Cites pop-up menus to movable menus of Wiseman, Xerox Alto 1976. PenPoint, GO, PenWindows, Momenta.
Graphical/virtual/synthetic keyboard on-screen keyboard on Apple MacIntosh, usable with mouse without physical keyboard. Also includes predictive type-ahead. Main use is as accessibility technology, combined with dwell-mouse for handicapped use.
Wacom tablet integrated into MacIntosh (?). Indexing of electronic ink notes (automatic concordance) by ink matching of stroke groups separated by distance, then matched by dynamic time warping. Both text (handwriting) and special symbols (weather maps) -- no reference to indexing of sketch input. Used as indexing/annotation method in Marquee video logging.
Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages.
Wintab also mentioned in Windows95/W4PC/TabletPC documents.
Also exists in version from about 2003 with minor updates from Wacom, later version 2010 (see file)
DataPen hand-held text scanner. Functional, in original packaging. Compare with Anoto pen.
Cited in Arai 1997 paper for hand-held optical digitizer: appears to be a text scanner. No copy, file is alternative references.
Hand-held computer -- format is like hand-held terminal, running Windows CE, touchscreen used as mouse replacement.
Optical image detection of finger pointing instead of mouse, using color mapping for detection. Mouse click by tapping shift key on the keyboard. Pointing mode for finger tracking by recognizing "pointing gesture" of hand.
Technology review of touchscreens: flexible resistive film (a la Elographics), Capacitive, Capacitive zone, SAW surface acoustic wave or SAW guided acoustic wave, IR infra-red touchscreens, strain gauge. Refers to problems with capacitive touchscreens. Mentions that acoustic SAW screens can sense multi-touch more than one point of contact. Capacitive proximity sensing (Microtouch zones). Does not use the word tablet. Cites Scriptel/Symbios electrostatic tablet with proximity sensing. Describes fabrication expense of rear-mounted electromagnetic digitizers. Electronic ink in windows 95.
PKCS11 API standard Version 1.0 for smartcards and other cryptographic tokens: single-user, mostly portable devices. Object-based in "C".
What it says: publics FTP servers on internet accessible by eMail.
Touchscreen sensor extends past edges of display for fixed buttons or keyboard. Touch sensor can be opaque on edges/margins. Compare with Wang Freestyle "The Guide"?
Handheld computer with slide-out/detachable keyboard: Mentions Momenta
Bibliography only: contains reference to Ledeen Recognizer in Newman/Sproull textbook, other interactive editors (without character recognition): Application GUI using Ledeen recognizer
Sales Automation application using Windows-based pen computer. Input into "letter guides", boxed character recognition input, makes distinction between character cells which are empty, and those which contain a space. Compare with Ward patent on simulated keyboard input using handwriting input.
Automatic software agents recognize text, but also figures, tablet, paragraphs of interest, via detectors and add information links (actions).
Touch panel displaying different pop-up menus depending on where it is touched: entry is selected by separate button when desired item is displayed.
Rotatable handwriting input widget/panel so that user can write/input from various directions, separate from orientation of touchscreen display. Compare with SIP panel?
Display can be rotated to laterally elongated position (landscape mode) or vertically elongated (portrait mode), control unit sets display format for windows. Saves orientation information when turned off. Compare with Radius Technology, Masimo 93?
Elastic matching (degree of one-dimensional deformation) with template for OCR optical character recognition. Mentions real-world preprocessing problems of difficult binarization, difficult separate from background. Samples taken from standard NIST OCR database.
Trapezoidal/rotational editing of part of a picture by defining four points of a trapezoid, and moving the four corners.
Optical mouse in pen/stylus form: light sensor in stylus, two colors of lights (or two filters), surface has grid of horizontal lines and vertical lines of different colors so that stylus/mouse can detect movement in X and Y by counting lines. States that can be used for writing (unlike mouse) because can detect movement in proximity, like many tablets.
Parsing of informal structure in electronic ink sketching in an electronic blackboard (whiteboard) system.
Case-study of designing touchscreen user interface for telephone, IBM project for pen-computing on a phone. Shows extensive use of paper mock-ups (sketches), simulation on Hypercard. Refers to notebook user-interface, rotate horizontal/vertical landscape/portrait orientation, visually similar to PenPoint (tabs, pen/stylus, etc.)
Touch sensors on keys of keyboard: user gestures (combinations of swipes) on space bar to enter/leave graphics/pointing mode for other mode. Mentions using two pointing keys for two hands: compare to Buxton bi-manual systems, multi-touch. Handicapped use by operating sensors with tongue. Grid array of sensors, scanned by rows and columns: optical touch sensor measures proximity of reflected light from small emitters: interpolation of signal intensity for higher resolution.
Elastic matching (see Tappert) applied to OCR of handwritten numerals, after thinning and line tracing.
Business tutorial on wireless, cellular, and other technologies: includes notes on EO/Hobbit pen computing, IBM Simon, PDAs with wireless communications
Textbook, historical reference and tutorial on cryptography: includes C source code on MD5, IDEA, DES, etc. Listing of patents p. 447 for RSA and Diffie-Hellman key exchange
email@example.com: long bibliography of pen/handwriting input and other GUI technologies
Rufus language: Indexing of document (for retrieval) by parsing for implicit or semantic structure in the document: d.g. Jimmy Carter in Title, rather than Author or mentioned in passing.
Electrostatic/capacitive digitizer with cordless stylus or finger touch: mentions shielding between LCD and sensor (front-mounted), greater optical transmissivity due to optical properties of ITO shielding compared to touchscreens. Cordless stylus requires battery. References Windows 95 with Pen Extensions (Pen Windows). Joint product with NCR?
Electrostatic/capacitive digitizer with cordless stylus or finger touch. Describes capacitive sheet digitizer as "non-contacting resistive digitizer". Has good simplified description of resistive sheet tablets, ratiometric / normalizing coordinates.
Symbios manufacturing hardware controllers for Scriptel transparent tablet digitizers: Cites advantages of proximity sensing, no electromagnetic interference, cordless stylus over other PDA touchscreen technologies. Simulationeous pen and finger-touch detection: multi-touch.
Pointer device (touchscreen/stylus) movement in different directions to adjust contrast up/down and brightness up/down. Compare with marking menus? Initiated by key, or active while a button is held down.
Optical digitizer on "smart paper", with a (passive) pattern printed on the paper -- Compare to Anoto technology. Appears to use a something similar to a circular barcode / ringcode: provenance not given, might be interesting to find. Digitizer tablet "smart paper" that puts a printed code on paper, position is read by optical sensor in the stylus tip. Compare with technology by Anoto et al. Shows circular barcode optical target, doubled 1d Barcode target, square matrix target for "smart paper" Two styli pictured, no parallax correction (?) to optical image in optical digitizer
Haptic/tactile feedback to a mouse, joystick, or other pointing device based on storing the window handle or other information in the input device and generating the feedback locally: faster than basing it on the information sent to the host
Handwriting recognition on tableet with editing: editing symbols == editing gestures for text. Handwriting edit gestures of down-left, pigtail, etc. -- compare with PenPoint and Pen Computing gestures.
Visual programming: Semantic constraints may interfere during editing of diagrams / sketch editing / Visual languages, as editing may leave state inconsistent: proposes incremental validation of constraints.
Prograph graphical programming language.
Electronic organizer (PDA) with full keyboard, touchscreen. Fax modem. Touchscreen four-corner calibration. Touchscreen used as mouse (menus etc.), electronic ink. Some touchbuttons duplicate keyboard functionality e.g. "Store" button same as "Enter" key. Pop-up onscreen numeric keypad. VT-100 terminal emulation. Mentions emulation of PDA on PC display.
Sharp IQ-9200 PDA, dates approximate (1995)? has full keyboard in clamshell design, touchscreen with stylus for sketch input.
"spatial parser" to attempt to recognize implicit structures in how people lay out information: example is paper notecards, organized as horizontal and vertical lists, stacks, composite (list with additional "title card"), heaps. Compare with visual languages and 2D/two-dimensional parsing of mathematical expressions.
Encrypted container for protecting information rights (digital rights managements DRM). Specifications for DigiBox container (license server DRM) submitted to ANSI IISP Electronic Publishing Task Force EPUB, WG4. Models / language include purchase versus pay-per-use (metered). Clearinghouse for usage information and payments. Attached header includes transport key block, container table of contents. Encrypting 10% ten percent of video file makes it effectively uncopiable. Compare with ContentGuard, Stefik. Cites to superdistribution.
In a Wellner-type optical/augmented desk / touchscreen, a tag with a button and an LED flasher. The LED position can be determined in the camera image, the LED is modulated to send a recognizable ID, and the button can be used as user input. Used to tag physical objects so that they can have augmented virtual capabilities (e.g. tag on phone ties it to on-line dialer director).
First (?) version of OpenGL graphics library: includes subpixel drawing, alpha blending for transparency., Screen-door transparency == cheesy transparency
Using pixel data to render transparent/overlay images in windows and objects, according to Z level of the windows or objects.
Touch sensor that makes 3D images of surfaces: rather than a thin sensor membrane, uses a tank or container filled with a liquid with a conformable membrane on top: pressed on an object, membrane with liquid conforms to shape of surface, video camera captures image of reflected light depending on angle of incidence with the (conformed) surface.
Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode
Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer"
Forms entry in handwriting recognition user interface, where a data-entry form field window pops up for entry, after entry of a pre-determined command using command/gesture symbols
Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode
Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer"
Sklarew/Grid very specific to resistive digitizer, no proximity sense (?)
Tutorial on pre-multiplied and not pre-multiplied alpha blending, producing a shaped image with partial transparencies. Treatment of "over" operator (transparency) in Porter paper of 1984. Alpha is a fourth value "transparency" or "opacity" in addition to RGB values. Discussion of pixels modeled as points, or as square windows to which an object may partially contribute by crossing part of the square.
A pixel is a point sample, not a square or rectangle. Must represent contribution of all graphics that are behind it: may be overlapping circle with region of adjacent pixels, or of broader nearby pixels for reconstruction of image. Argument applies also to scanning and printing. Should not refer to shape of pixels (e.g. rectangular, non-square) but to non-square/non-uniform spacing of pixels.
Alpha blending: integral alpha, premultiplied alpha, digital printing, graphical compositing, matte / masking / matting blending. Traces history to analog-film compositing using a physical matte to combine images. History of blue-screen/green-screen matting, Petro Vlahos. Digital compositing an obvious simulation of known analog-film techniques, but digital matting (using alpha blending) not obvious.
General review of electrostatic/capacitive sensing, from simple proximity detection up to 3D and beyond. How to determine position using sensors in any particular configuration -- basically calculating or determining a correction table or formula to get the position of the object from the mix of sensor values, for whatever configuration you know the sensors are in. No references to commercial products e.g. Scriptel, academic references.
Self-4.0 user interface: object-oriented programming originated in attempt to make software model real-world behaviors, need to apply those principles more closely. reification of layout constraints (automatic layout) for user interface, composition through embedding. Software can be modified while running. Slots for actions on an object. Graphical objects referred to as morphs. Cites to Visual Basic, but not to PenApps by Slate.
Description of Newton application and OS architecture: slots, packages, interpreted bytecodes, no separate address spaces for different applications. Describes Newton as simplified version of "Self", frames take place of Self objects. Defines View system (Smalltalk 80), and MacApp. Newton applications delivered in packages, containing parts dispatched to the appropriate part of the OS when installed: e.g. application part, font use by application, special communications driver. Pages in virtual memory are compressed, and decompressed when loaded. Picture from Newton Toolkit: cites MacApp for views.
Discussion of NewtonScript for the Newton pen-computing PDA, based on prototype-based inheritance language SELF. Note: NewtonScript adapted from SELF's "parent slots". Self based on SMALLTALK.
NewtonScript prototype-based language for Newton development: describes historical antecedents. Visual Pen development environment for Newton Toolkit: view system and collection of slots. Browser weak at supporting non-UI code, really just a "frame hierarchy" editor. Inheritance (packages?) inspired by HyperCard. Also cites to Bay-Wei Change, 1993, Smalltalk-80, Self
Tutorial on Visual Basic language and development environment
WriteTouch electrostatic/capacitive digitizer from Scriptel supports either finger or stylus input: can do multi-touch? Announcement of cordless electrostatic digital pen using Symbios Logic mixed signal design with Scriptel.
References, Product information on Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator": hands-free cellphone option, touchscreen, but virtual keyboard only, no handwriting recognition. Magic Cap OS from General Magic.
Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator"
Study and statistical model (based on frequency digrams) of virtual keyboard typing speed on touchscreen with stylus. States more accurate and faster (overall) than handwriting recognition. QWERTY keyboard. Notes slipperiness of touchscreen, visual parallax. No citations to commercial pen computing systems (PenPoint, PenWindows, etc.)
Force/touch touchpad sensors on desktop/tabletop, may be used to create virtual input devices (virtual keyboard, menu, tablet, etc.), sense location of physical objects (phicons?). Identify/authenticate user by force-image/biometrics of palmprint, etc. Instrument/augment objects such as locks, floor.
Technical dictionary of computer terms 1994
Sketch recognition for mechanical design: uses models of physical/mechanical constraints to produce actual designs from sketch. States modifies sketch until a design that functions in a specified manner.
Cited in Arvo00
Electrostatic resistive sheet touchscreen digitizer, can sense both a stylus by a particular signal and a finger touch by capacitance at the same time: e.g. write with stylus while selecting menu with finger. Gives formula for shift and scale correction/calibration, but not quadrilateral calibration correction. multi-touch capable?
Semi-transparent "filter" viewer window can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text.
Semi-transparent "filter" viewer window has VOR "Viewing Operation Region", can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text.
MagicLens: visual transformations of part of a lower window when a second window is over it. Gives the effect of the second window being a lens: e.g. shade the color, magnify, show hidden lines/wirefram, show streets on a map, etc.
Cordless digitizer pen, sense stylus buttons by button changing resonance frequency of stylus.
Remote whiteboard touchscreen system combining LiveBoard and networked personal computers.
Open Windows / X Windows reference manual: includes vkbd(1) virtual keyboard/soft function keys display manager: display labels of function keys on screen with application-dependent labels automatically -- can use physical function keys or on-screen virtual keys input area.
Open Windows / X Windows reference manual
Optical touchscreen, LED light emitters and photodetectors are arranged diagonally instead of X/Y. Light intensity reduced on corner emitters to reduce reflections.
Marking menus with stylus: animation feed back to user if mark (inadvertently) triggers a command gesture. Compare with press-and-hold, and stroke if do not wait.
Aviary system: Re-mapping of UI events (mouse button clicks) to other application events (or system events). Compare to press-and-hold event mapping in later Windows version.
Report on StarFire video project, showing speculative multi-touch user interfaces. Primary concerned with virtual video prototyping by means of "special effects" dramatized video production, which forces one to consider actual use cases in the course of building a shooting script for a film.
Essays on user-interface design, mentions 1994 "StarFire" video speculating on future touch-input portable computers, including a demonstration of a multi-touch (bimanual/two-hand) zoom gesture. Discussion on handwriting only mentions Graffiti by Palm. On touchscreens, states that poor accuracy (or visual parallax) causes users to slow down in an attempt to be more accurate: speculates on fiber-optic coupling plate (?) to reduce visual parallax.
Hardware user's manual for Toshiba T200 and T200CS pen computers -- mentions screen rotation, handwriting, etc. but does not name the operating system (PenPoint or Windows for Pen Computing).
Physical operational unit of Toshiba pen computer running Windows for Pen Computing 1.0 on top of MS/DOS 6.22. Photographs of screen rotation between landscape and portrait orientation, on-line gesture tutorials (with examples), Pen Palette pop-up handwriting bar for text entry, virtual keyboard in multiple configurations. SideKick 2.0. Base is Windows 3.1, with pen extensions.
Similar unit to Toshiba T200 80 Pen Windows 1.0, but color screen (better contrast on physical device and in photographs)
DataGlyphs: see other references. Compare to Anoto. Shows square matrix optical target, with asymmetric halftone dot patterns in cells of a digital halftone image
Mentions papers and work direct manipulation / pen / stylus interfaces for mathematical input and editing (pp 27-30)
Marking menu: get a pop-up of a rotary/pid menu with letters/characters, arranged concentrically in multiple rings, or set of characters is magnified in the direction you move the stylus from the center. Hand occlusion / obscuring of menu dealt with by showing pop-up menu at an offset from pen tip. Audio feedback (clicks) on the flick gestures at start and end. Visual feedback of selected character and idealized form of the flick gesture. Target can also be displayed continually, like a floating graphical keyboard.
Installation instructions for Apple T-Cube pie-menu (unistroke handwriting) text input.
ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. Uses Unistroke character recognition, click/tap gestures on heirarichal menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for left- or right-handed operation.
ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. "Tab" refers to "small tablet computer" -- touchscreen computer. Uses Unistroke character recognition, click/tap gestures on heirarchical menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for left- or right-handed operation.
Ultrasonic vibration of a few micrometers amplitude makes surface feel slick, less frictio, due to small air film (hypothesized) under finger. Haptic feedback of stickiness, roughness, small protrusion for touchscreens, other surfaces.
Real-time electronic ink annotations while video is being made, uses as keys to video. Intent is to eliminate having to review the video later to log/annotate/index it. User takes notes, can circle a "keyboard" in the notes to use as index. Compare with Anoto smart-pen note taking (combined audio/electronic ink recording). Horizontal scratch-out gesture (not confused with M/W in handwriting recognition): keywords translated later manually by user typing?
Newton OS, much is on details of memory management Memory space is a single (large) linear address space but mapped in pages, protection among tasks: because not context mapping among multiple address spaces, thread/task switching is very fast, object IDs are just their linear address. (Novel at the time?) downloading of system updates, other software.
Mentions Freestyle, and asserts that people "like" paper compared to electronic documents. Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Mostly a user-interface experiment in combining electronic simulation with real paper documents. Image subtraction and motion detection to locate hands etc. (Longish discussion of problem using a standard "C" image processing library from Modula, due to garbage collection).
LabVIEW Graphical programming language "G" for virtual instruments: has simulated mechanical action of boolean switches, both switching and latching (when pressed, when released, until released). Example of Global Switch.
About hand and body gestures of the form "put-that-there" gesticulation, rather than handwritten gestures as part of a computer GUI: refers in contrast to pen-based gestures with hand-held objects, instead of open-hand gestures (topic of thesis)
Bibliography of gesture recognition research: primarily about hand and body gestures of the form "put-that-there" gesticulation, rather than handwritten gestures as part of a computer GUI
Panel discussion on gesture recognition research: primarily about hand and body gestures of the form "put-that-there", rather than handwritten gestures as part of a computer GUI. Edward Altman, Jelle Coutax, Sidney Fels, Wen Gao, Nobuo Hataoka, Bernard Hibbits, Volker Kuehn, Christoph Maggioni, Axel Mulder, Polly Pook, Alan Wexelblatt.
Gesture recognition for natural gestures (whole body gesticulation, empty-handed gestures/gesticulation), rather than pen-input gestures or handwriting recognition
Multi-user information system, to use cordless pens, simultaneous tracking of multiple pens: 2..3 users enough in practice. Prototype uses resistive film with corded pen. Text handwriting recognition using CIC MacHandwriter. Wall mounting of prototype more awkward than original MacHandwriter hardware with desktop tablet. multi-touch with styli.
Simultaneous voice and electronic ink / handwriting recording in portable device: indexing of handwritten notes to audio recording. Compare with Wang Freestyle (cited) and to Anoto SmartPen/LiveScribe (later).
IBM Simon early "smartphone" with touchscreen. ROM-DOS operating system. GUI for specific applications "Navigator". Compare with EO/PenPoint with cellular phone?
Dylan: Competing language and IDE to NewtonScript for the Newton PDA: IDE released as technology release in 1995
General Magic development of pen-based PDA/pocket digital communicator, using Telescript programming language to implement software agents. Compared to Apple Newton. Bill Atkinson, Andy Hertzfeld.
Resistive film digitizer, voltage gradient. Correction for edge / pincushion distortion by using a variety of connectors at the edges, with resistance corrections on the connectors.
Secure network sockets: Secure Network Protocol in distributed security, predecessor to SSL key exchange and public-key encryption
Scrollbar display highlight marks corresponding to where text had been found in a document. Compare with Russian patent to Gorbunov on the similarl feature.
Xerox PARCTAB palm-sized mobile computer / PDA, worn like a pager. Three function buttons and stylus user interface on display touchscreen.
Secure processors: (co)processors with secure code in ROM, processor packed in epoxy with self-destruct lines, checksum on all loaded code. Compare with smartcards. See also Dyad system (secure DRM).
Integrated touchscreen/display device, connects wirelessly to Windows PC (Windows 95, Windows 3.1, PenWindows?). Image from billbuxton.com, 2012. Electronic file has additional references.
Recognize dwell/press-hold in stylus and distinguish between mouse / gesture / handwriting input mode: press-and-hold
Silk Cursor using "silk stocking" effect: partial transparency to simulate (non-continuous) difference in depth instead of stereoscopic views.
Handi: handsketch-based diagram editor in Express-G language, user-defined an pre-programmed unistroke gestures: one-stroke "T", circle, square, semicircle left/right gestures
Human touch sensing on touchpad and other objects (furniture, location in room) using capacitance: field shunted to ground by human body, and field transmitted through conductive human body to stationary receivers. General remarks on capacitive sensing in general. Used to implement capacitive touchscreen ("finger-pointing mouse"). Can distinguish multiple persons (multi-touch)
biblio93 (Prev page)
(Next page) biblio98
This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015.
I can be contacted at firstname.lastname@example.org.