Last week a patent application from Google was published in Europe that was filed for in August 2023. The patent covers future users being able to write with an electronic stylus on multiple surfaces such as a desk, a notepad, a clipboard and even in the air and be seen by the user wearing smartglasses.

In Google's patent background they note that users may desire to write with an electronic stylus. However, capacitive surfaces may not always be available to detect movement by the electronic stylus.

Generating Writing based on Movements of the Stylus

Google's patent states that a user can write with an electronic stylus on a surface, such as a table, a notepad, or a clipboard. The surface does not need to be capacitive, electronic, or have any means of detecting movement of the electronic stylus.

A computing device generates two-dimensional writing based on measured three-dimensional movements, such as multiple three-dimensional accelerometer measurements, of the electronic stylus. The measured three-dimensional movements can be received by the computing device in a series that is ordered based on times of measurement. The electronic stylus includes an accelerometer that measures and transmits measurements, such as three-dimensional accelerometer measurements, to the computing device.

 In some examples, the user makes writing motions with the stylus without the stylus contacting a surface, such as by writing into the air.

In some examples, the user writes with the stylus alternating between contacting a surface and not contacting a surface.

The computing device can generate two-dimensional writing based on measured three-dimensional movements with or without contact by the electronic stylus with a surface.

The computing device can include a head-mounted device (such as smartglasses or goggles that create an augmented reality environment), a laptop or desktop computer, or a smartwatch.

A technical problem with generating two-dimensional writing based on measured three-dimensional movements is that noise or jerky or irregular movements of the electronic stylus can cause undesired marks within the two-dimensional writing. For example, a portion of the stylus that includes an accelerometer and/or gyroscope (which measures the three-dimensional movements) can move while an end portion of the stylus is stationary with respect to the surface, or movement of the portion of the stylus that includes an accelerometer and/or gyroscope can be disproportionate compared to movement of the end portion of the stylus with respect to the surface.

A technical solution to this technical problem is for the computing device to determine a symbol that is being written and correct, modify, or augment the determination of the two-dimensional writing based on the determined symbol.

The computing device determines a symbol, such as a letter or a shape, that the user is writing. The computing device can determine the symbol based on the measurements of the user drawing or writing a portion of the symbol, based on a prediction of the current symbol that is based on one or more previous symbols (such as based on a Markov chain), or combinations thereof.

Based on the determined symbol, the computing device can modify portions, such as strokes, of two-dimensional writing that are included in the symbol. The computing device can, for example, smooth out, or remove, some of the measured movements based on strokes that make up the determined symbol. A technical benefit of modifying portions of the two-dimensional writing based on the determined symbol is that the computing device can thereby present the writing in a style that the user wrote the writing in, rather than in a predetermined font, without some of the jerks measured by the electronic stylus.

The Lead inventor behind Google's European patent 2025034229 is listed as D. Shin, Senior Software Engineer at Google DeepMind).