_%E5%B7%A5%E4%BD%9C%E5%8D%80%E5%9F%9F%201.jpg)
_%E5%B7%A5%E4%BD%9C%E5%8D%80%E5%9F%9F%201.png)
BLIND SHOPPING
Blind Shopping is a guiding system for the visually impaired to become independent in shopping. With the technology of positioning and bone-conduction, the visually impaired can enjoy shopping just like normal people.
Information
Personal project
Field
Assistive device
Duration
2017
Task
User research
Problem framing
Data collection
Concept development
User flow
Prototype
Research
Because of the curiosity about whether there is a hidden problem in a grocery store we usually go to, I created a user journey table to analyze the whole grocery shopping behavior. Finally, I gained insight from the table below.
To identify their difficulties and expectations, I volunteered to assist visually impaired people in Taipei light house dedicated to improving the quality of the visually impaired’s life in Taipei City.
While assisting them and observing other volunteers performing shopping cases, I got various feedback and insight from visually impaired people, the organization, and other volunteers.
This shows that “seeing” accounts for a large proportion of grocery shopping behavior.
Problem
From this, it also can be seen that people with visual impairment(PVI) are excluded from the grocery store, due to their sight loss. Shopping must be a challenge for them!
How to allow the visually impaired people to enjoy grocery shopping as much as the general people?
Field study
To identify their difficulties and expectations, I volunteered to assist visually impaired people in Taipei light house dedicated to improving the quality of the visually impaired’s life in Taipei City. While assisting them and observing other volunteers performing shopping cases, I got various feedback and insight from visually impaired people, the organization, and other volunteers.
Feedback
Visually impaired people |
˙Difficulties in finding goods, reading product labels and walking in an environment
with many barriers such as unknown places or a street.
˙Prevent from being overly dependent on other people.
˙Enjoying the freedom of shopping by themselves.
The organization |
˙Becoming more short-handed from day to day.
Volunteers |
˙Difficulties in working around visually impaired people’s schedules.
Insight
˙Assisting visually impaired people in becoming independent is not only a
good solution but also helpful to their self-esteem.
˙A huge challenge they face during shopping is navigation and product
identification. Therefore, it is important to establish the right equipment to
help them safely navigate from one place to another and identifying objects
on the shelves without any support from people without disabilities.
Literature review
What are the existing approaches to address the problem in navigation and product identification?
Researchers have developed multiple solutions to help visually impaired people navigate and identify products.These solutions are divided into two categories:
(1)Tag Based Systems:
Such as RFID, NFC, and Bluetooth which use wireless components to transfer data from a tag attached to an object for the purposes of automatic identification and tracking.
(2)Computer Vision Based Systems(CV):
Some of these systems require unique visual tags such as QR code, Barcode, AR markers to be placed on products for detecting and giving PVI all available details about the products. Other systems do not require tags to be placed on products. Instead, they utilize information about the objects’ physical features to identify them.
Insight
However, because each environment has specific features, individual systems are not suitable under all situations. It seems that hybrid systems by a combination of two or more systems can balance trade-offs, exploit the main advantages, and avoid the disadvantages of individual systems.
Concept
Combine bluetooth and CV techniques to help visually impaired people safely navigate their way, find specific items, and make purchases.
The system was designed into two main systems: portable front ends(smartphone, smart glasses, white cane) and clould-hosted back ends(iBeacon, server, and database). The database contains product detail data and shopping maps.
In addition, whenever they want to know chosen product’s details, they can scan its barcode with the white cane. The barcode will be processed and matched with its product and convey all the information to the glasses through the smartphones.
When a navigation demand occurs, the smartphones will detect iBeacons in ranging area, send each iBeacon’ information to the server, receive the direction of the shortest selected path from the database and convey it to the white cane and the smart glasses to navigate visually impaired people.
˙Assisting visually impaired people in becoming independent is not only a good solution but also helpful to their self-esteem.
˙A huge challenge they face during shopping is navigation and product identification. Therefore, it is important to establish the right equipment to help them safely navigate from one place to another and identifying objects on the shelves without any support from people without disabilities.
After listening to the information, they can decide whether to buy the product. If they say yes, the product will be added to the virtual cart. At the time of leaving the store, the iBeacon device in the store will post the purchase to their mobile payment account.
Visually impaired people |
˙Difficulties in finding goods, reading product labels and walking in an environment with many barriers such as unknown places or a street.
˙Prevent from being overly dependent on other people.
˙Enjoying the freedom of shopping by themselves.
When a navigation demand occurs, the smartphones will detect iBeacons in ranging area, send each iBeacon’ information to the server, receive the direction of the shortest selected path from the database and convey it to the white cane and the smart glasses to navigate visually impaired people. In addition, whenever they want to know chosen product’s details, they can scan its barcode with the white cane. The barcode will be processed and matched with its product and convey all the information to the glasses through the smartphones. After listening to the information, they can decide whether to buy the product. If they say yes, the product will be added to the virtual cart. At the time of leaving the store, the iBeacon device in the store will post the purchase to their mobile payment account.
10
9
8
7
4
6
5
3
2
1
11
12
13
14
15
Working prototype
With 3D printing and a LED module, I made a prototype of a white cane that can demonstrate the size, appearance and lighting.
Final design
Vibrator
directs movement(Front, back, left, and right) for warnings or providing direction commands to the PVI
Camera
assists PVI in recognizing the
products on the grocery shelf
Camera
assists PVI in identifying the underlayer of the product without squatting
Scan button
controls the up and down camera to locate the desired product from the grocery shelf, scan its barcode and get details
Vibrator
directs movement(up, down, left, and right) to find the desired product
EPE(Expand able poly ethylene)
makes PVI feel comfortable to hold
Shaft
keeps wrists straight to avoid making cumulative trauma disorders
Warning light
automatically adjusts light output up and down by photoelectric lighting control to warn other shoppers or drivers of possible danger or problem
EPE(Expand able poly ethylene)
makes PVI feel comfortable to wear
Bone conduction transducer
provides sound clarity in a noisy environment
Microphone
receives the voice command from PVI
Camera
assists PVI in detecting
obstacles in their path
Volume button
adjusts the volume(L:turn up/S:turn down)
Assistance button
allows PVI to interact with the white cane and smart glasses by speaking
Control key
helps PVI select what product
detail they want to listen to
Power switch
turns on or off the smart glasses
User flow
How will visually impaired people use BLIND SHOPPING during the whole shopping process and enjoy the freedom of shopping by themselves!
iBeacons present the information to your device based on the context of where you are. As you are near the store on the sidewalk, you will receive a short little message to guide you to enter the store.
Upon entering a store, a consumer’s payment app identifies the inside iBeacon signal and alerts the merchant’s POS system of the user’s presence so that you can tee up a pre-approved digital payment method for whatever you may buy.
As you approach the junction, iBeacon installed on traffic lights reaches out to your smart glasses and white cane to tell you when to stop and go. Thus, you can feel safe whenever walking to the store or going back home.
iBeacons present the information to your device based on the context of where you are. As you are near the store on the sidewalk, you will receive a short little message to guide you to enter the store.
More projects
_%E5%B7%A5%E4%BD%9C%E5%8D%80%E5%9F%9F%201.jpg)