We're working to make use of the awesome technology in the newly released Leap Motion to help improve children's education.
Francisco Guzmán (@fguzman) ~ Design Lead
I'm a coterm in Symbolic Systems with a focus in Human Computer Interaction. I have an interdisciplinary background - my undergraduate degrees are in MS&E and STS. I've been involved in a few different areas through internships, including product management, strategy & operations and more recently roles in UI/UX design.
Amanda Lin ~ (@pandolin) ~ Development Lead
I'm a senior in Symbolic Systems senior a coterm in CS, specializing in HCI. Last summer, I worked as a Product Management intern at a startup in SF, and the summer before that, worked as an intern on the Developer Relations team at Apple.
Vivian Shen ~ (@vivianshen) ~ Evaluation Lead
I'm a junior majoring in CS focusing on HCI. I've worked as a software engineer on the Ads team at Google in the past and done research at the CS Security lab at Stanford. On the business side I've done business development for a mobile advertising startup and also worked on consulting projects for tech-focused companies.
Tim Shi ~ (@timgshi) Project Manager
I'm a senior in HCI and coterming in HCI as well. I'm interested in product management and have a diverse background in engineering and business, with internships at Google and Nike in the past. At Google I was an iOS engineer on the Google Search App and at Nike I was a strategy consultant on Access to Sport.
Our focus is on the needs of children. We have conducted needfinding exercises in three locations: nursery/schoolrooms, children's hospital waiting rooms, and local parks. Our preliminary focus is on creating a more interactive experience in these locations, and helping children interact with one another without being intrusive. We want to satisfy their need for an engaging and stimulating activity (that's preferably educational, for the sake of their parents) while encouraging them to have fun with each other as well.
Photos of initial brainstorming:
As suggested, we went for quantity with our initial brainstorm. We considered many different ideas because of the many areas in which we are interested in helping children - such as the park, nursery and children with learning disabilities.
Three promising approaches:
First test This first video was a way to test the technology we are using, the Leap Motion. In the video, we try a variety of finger movements in order to get an understanding of the detecting capability. This first test served as a pathway for us to explore some of the other motions in the videos below.
Balls This video is a continuation of the testing and the different features of the Leap Motion - this testing helped us understand the range that the device has, which informs the way we think about potential prototypes.
Stirring! This testing video was inspired by one of the implementations we proposed during our skit - cooking mamma. Educational games are still of strong interest to us, and so we wanted to experiment and see how feasible some of the interactions we'd like to incorporate are.
Four hands! In our observations we've noticed how important social interaction can be in enabling other types of play between children and their parents. This test was conducted to see how many inputs can be detected - again, as a way to understand the realm of possibilities in the design of our applications.
Two Video Prototypes
Building blocks Physical play has been shown to be instrumental for a child's play and especially at an early age for cognitive development. When thinking about the way children play, especially children with disabilities that prevent them from fully grasping physical objects, we wanted to simulate an experience that can lead children through that process in ways of interaction. This video prototype is a simple way to show our main idea - the child would move a digital object through gestures, but still understand the repercussions that layering objects can have - learning the fun in building. We feel this would be especially helpful, as was mentioned before, for children who have difficulty grabbing physical objects.
Karel! There is a strong push in education to better expose children to STEM fields at an earlier age. From some of our observations, we saw that many teachers are playing around with ways in which they can achieve this - especially for younger children who's capabilities may not be advanced enough to grasp algebra or computer science, for example. There are small things we can do, however, to put in place the building blocks for that learning. This video demonstrates Karel moving through a maze in order to get to the star. The child would break down the actions, one by one, that Karel has to take in order to reach the star - this is a way of establishing the basic thinking behind computer science.
One Sketch (not related to Leap Motion - but a potential idea to continue to ponder on)
We came to class on Thursday 2/14 ready to Wizard of Oz test our gestural Karel the Robot prototype and had several users test controlling Karel using gestures. The prototype involved using hand gestures to move commands ("turn left", "turn right", "move forward") from one side of the screen to the other in order to form a "program". Users saw Karel in a maze and were prompted to help guide him to the exit by building a program and clicking the run button.
As we tested with each participant we interviewed them to get a sense for how intuitive our prototype was and learn how they felt children might perceive the game. Through our tests, we heard multiple times that this idea might be too abstract for 3-5 year old children and that the need for a gestural interface was not clear, i.e. a simple arrow key interface would be just as effective. Based on this feedback we decided to scrap the Karel idea and head in a new direction.
On our visit to the Bing Nursery we learned that the educational units taught there are driven by student interest. Teachers test and gauge interest in various topics by laying out objects on the "discovery table" and watch the students to see what might be most popular with the kids. We plan to prototype a tool to aid teachers in enhancing the discovery table by providing a gestural interface that lets students interact with various potential unit topics through a prezi-like visual. This system will be able to track and gauge early interest in subject areas for the educators before they make the effort to develop a unit.
In creating our enhanced, digitized discovery table, we knew we had to include features that would make it both intuitive and easy for children of such a young age to use. First, we decided to include all of the children's photos on the initial login page. This way, each student can point to his or her own photo in order to "log in." After much brainstorming, we decided this would be the easiest way for teachers to be able to track which student was interested in what subject. All of the child's actions after logging in are stored in our database, with information that will be provided to the teachers every time they want
Next, we wanted to implement a way of easily navigating through a series of topics, so we decided to include a number of different topics on the initial page after logging in. Each of the topics has a photo and a title, because during our observations we noticed that the kids could not yet read. The students can then point to any of these titles in order to reveal more photos related to the topic. Meanwhile, we measure how long kids spend on each topic and store that information in our database so that teachers can see which topics the student was most interested in. We will also aggregate all of the time spent on each topic so that by all of the students so that teachers can gauge which topics had the most interest.
Ultimately, we wanted to add in these features to 1) ease the process for teachers at Bing for finding new topics to teach, allowing them to do a "discovery table" without committing to first purchasing physical objects related to the topic, and 2) make it so other schools can implement this same idea of a discovery table in order to create a curriculum for students.
Though we originally started out focusing on kids at Bing Nursery, we realized that the age demographic there was too young to fully take advantage of the Leap technology. We ended up pivoting and using our prototype for elementary schoolers instead.
Our motivation was to create a fun educational experience for kids and keep them engaged in educational content. A lot of times textbooks can be very 2D and un-interactive, so we wanted our application to be a way for kids to learn dynamically. Our end product is essentially like a digital textbook with fun content and gestures that keep them engaged and provide them with immediate feedback on the content they're learning about.
Some of our driving questions included: What is the best age demographic for these kids? What needs can we address that their current educational materials can't? Can we help keep kids more engaged in the educational content than before? Will the children be able to easily pick up the technology and use it?
We hypothesized that the novelty of the Leap motion software would draw kids in, as well as the interactive content of our application. We were a bit unsure of how adept the kids would be at picking up the technology, but we hypothesized it would be a relatively quick learn for them because a lot of them have exposure to iPads etc. already.
We will bring in 3 laptops and 3 Leap Motions into the elementary school that we are testing at. We will set them up in the library, and during lunch when kids come into the library they can volunteer to use the application if they wish.
We will have the kids queue up to use the leap one by one, and let the kids crowd around the computer to read the content together.
The tasks we will have them do include:
We will ask the kids questions about what they think they should do to advance through the pages, if the content makes sense and whether they find it interesting. The questions & things we want to observe are below.
Elementary school students (grades K-5) at Nixon Elementary School
Sent emails to the elementary schools in the area, had preliminary meeting with the principal to explain project and talk about best testing environment. Set date for bringing in the prototype and had teachers inform students that there would be a demonstration happening at lunch if they wanted to come and play with it.
We ended up having a really good turnout for our testing. We went to the elementary school two days in a row during lunch at the library, and had the teachers announce that we had come to demonstrate the game. The first day we had ~15-20 kids lining up to play the game, and we maybe had ~8 of them be able to play the game because we only brought one laptop. The kids were really excited about the gesture based nature of the game and we knew there was a lot of interest, so we decided to come back another day.
When we came back the next day, we came with 3 laptops & leaps and could manage the crowd better. This time, through word of mouth, we had more like ~30-35 kids show up to play with the game. All of them had a chance to use the leap briefly. We recorded observations below.
We learned that kids are really adept at picking up new technologies--within a minute or so a child could properly use the system and the children watching could all understand it as well. They also had pretty long attention spans and easily understood the point of the game and technology. The novelty of the gesture-based system helped keep them engaged with the educational content. We also found that the gesture-based system was good for keeping them entertained, but also had practical implications like being more sanitary than a touchscreen and being easier to manipulate than a mouse (which they said was painful after a period of time)
We found that they thought the picture based content was really engaging, and loved using the Leap for interacting with the content. They found the technology a cool way to make the content more interactive. They wanted as immediate feedback as possible-- they expected the find gestures to correlate with the speed of a mouse.
A lot of the kids were used to a certain model of technology (ie. clicking vs. hovering for selection) that our demo didn’t always parallel with. We’re going to change it so that our demo aligns a bit more with what they’re used to (see implications.)
Note: we couldn't take too many pictures because of privacy issues with kids, but we were allowed to take a couple if their faces weren't shown.
Second Prototype Features: