Here is the video of our talk at Strata+Hadoop World Conference 2014 in Barcelona (Courtesy O’Reilly Media, Inc.).
The Strata + Hadoop World in Barcelona 2014 Complete Video Compilation is also available at O’Reilly’s – and highly recommended.
Here are my slides:
So many things about wearable tech, self tracking, and the Quantified Self! So we started to put everything we read into a Flipboard Magazine to share with everyone.
Looking forward to your comments!
From February 18th to 20th, O’Reilly’s Strata+Hadoop World Conference will once again be the most important Big Data event of the year. More than 6,000 visitors are expected at San Jose Convention Center. I’ve been visitor to every Strata from the beginning. Thus I regard it as a special privilege not only to be there and listen, but give talk myself.
Here is the link to our session:
“Smartphone Data: Tell the Story of People’s Lives.”
Come and visit us there!
“Facebook would never change their advertsing relying on a sample size as small as we do medical research on.”
(David Wilbanks)
People want to learn about themselves and get their lives soundly supported by data. Parents record the height of their children. When we feel ill, we measure our temperature. And many people own a bathroom scales. But without context, data is little meaningful. Thus we try to compare owr measurements with those of other people.
Self-tracking has been trending for years. Fitness tracker like Fitbit count our steps, training apps like Runtustic deliver to us analysis and benchmark us with others. Since 2008, a movement has been around that has put self-tracking into its center: The Quantified Self.
Self-tracking has been tending for years. In this picture you see a wristband that already made it into a museum and is now on display in the London Science Museum.
However it is not just self-optimizer and fitness junkies who measure themselves. Essential drive to self-tracking originated from self-caring chronically ill.
In the US like in many countries lacking strong public health-care, it becomes increasingly common to bring self-measured data to the physician. With many examinations this saves significant consts and speeds up the treatment. With Quantified Self, many people have been able to get good laboratory analytics about their health for the first time ever. One example is kits for blood analysis that sends the measurement via mobile to the lab and then displays the results. Such kits are e.g. widely in use in India.
Also for family members and nursing staff, self-tracked data of the pations is useful. They draw a realistic picture of our conditions to those who care for us. Even automatic emergency calls based on data measured at site are possible today.
The image at the top is taken from the blog of Sara Riggere, who suffers from Parkinson. Sara tracks her medication and the syptoms of her Parkinson’s desease with her smartphone. Her story is worth reading in any case, and it shows all facettes that make the topic “own data” so fascinating:
http://www.riggare.se/ and
http://quantifiedself.com
Mood-tracking – a mood diary. People suffering from bipolar disorder try to help themselves by recording their mood and other influences of their lives. By doing so, they are able to counteract, when they approach a depression, and they are able to finetune their medication much better, than it would be possible by the rare visits to their psychiatrist. (Shown here is soundfeelings.com)
Self-recorded data for the first time maps people’s actions and condition into an uninterupted image. For research, these data are significantly richer than the snap-shots made by classic clinical research – regarding case numbers as well as by making possible for the first time to include the multivariate influences of all kinds of behavior and environment. Even if only a small fraction of self-trackers is willing to share their data with researchers, it is hardly to imagine the huge value the findings will have for medicine, enabled by this.
The difficulty with these data: they are so rich and so personal, that it is always possible to get down on the single individual. Anonymization, e.g. by deleting the user id or the IP adress is not possible. Like fingerprints, the trace we leave in the data can always identify us. This problem cannot be solved by even more privacy regulation. Already today, the mandatory committment to informed consent and to data avoidance impede research with medical data to such extent, it is hardly worthwhile to work with it, at all. The only remedy would be comprehensive legal protection. Every person sharing their data with research has to be sure that no disadvantages will come from their cooperation. Insurance companies and employers must not take advantage from the openness of people. This could be shaped similar to anti-discrimination laws. Today, e.g. insurance companies are not allowed to differenciate their rates by the insurant’s gender.
Another issue lies within the data itself. First, arbitrary, technical differences like hardware defects, compression algorithms, or samling rates make the data hard to match. Second, it is hardly the raw data itself, but rather mathematical abstractions derived from the data, that gets further processed. Fitbit or Jawbone UP don’t store the three-dimensional measurements of the gyroscope, but the steps, calculated from it. However, what would be regarded as a step, and what would be another kind of movement, is an arbitrary decision of the author of the algorithm programmed for this task. Here it is important to open the black boxes of the algorithms. As the EU commission demands Google to open its search algorithms, because they suspect (probably with good reasons) that Google would discriminate against obnoxious content in a clandistine way, we have to demand to see behind the tracking-devices from their makers.
Data is generated by the users. The users have to be heared what is made from it.
Our phones register in radio cells to route the calls to the phone network. When we move around, we occasionally leave one cell and enter another. So our movements over leave a trace through the cells we have been passing the course of the day. Yves-Alexandre de Montjoye and his co-authors from MIT explored, how many observations we need, to identify a specific user. Based on actual data provided by telephone companies, they calculated, that just four observations are sufficient to identify 95% of all mobile users. We need just so little evidence because people’s moving patterns are surprisingly unique, just like our fingerprints, these are more or less reliable identifiers.
When we analyze the raw data, that we collect through our mobile sensor framework ‘explore’ we found several other fingerprint-like traces, that all of us continuously drop by using our smartphones. Obviously we can reproduce de Monjoye’s experiment with much more granular resolution when we use the phone’s own location tracking data instead of the rather coarse grid of the cells. GPS and mobile positioning spot us with high precision.
Inside buildings we have the Wifis in reception. Each Wifi has a unique identifier, the BSSID and provides lots of other useful information.
Wifis in reception around my office. When the location of the wifi emitter is known we can use signal strength to locate users within buildings.
Even the aribrary lable “SSID” can often be telling: You can immediately see what kind of printer I use.
To provide compass functionality, most smartphones carry a magnetic flux sensor. This probe monitors the surrounding magnetic fields in all three dimensions.
Each location has its very own magnetic signature. Also many things we do leave telling magnetic traces – like driving a car or riding on a train. In this diagram you see my magnetic readings. You can immediately detect when I was home or when I was traveling.
The way we use the phone has effect on the power consumptions. This can be monitored via the battery charge probe:
The battery drain and charge pattern is very unique and also telling the story of our daily lives.
All the sensors in our phones have typical and very unique inaccuracies. In the gyroscope data shown at the top of the page, you see spikes that shoot out from the average pattern quite regularily. Such artifacts caused by small hardware defects are specific to a single phone and can easily be used to re-identify a phone.
“We no longer live in a world where technology allows us to separate communications we want to protect from communications we want to exploit. Assume that anything we learn about what the NSA does today is a preview of what cybercriminals are going to do in six months to two years.”
Bruce Schneier, “NSA Hacking of Cell Phone Networks”
As Bruce Schneier points out in his post: there are more than enough hints that we should not regard our phones as private. Not only have we learned how corrosive governmental surveillance has been for a long time, there are lots of commercial offerings to breach the privacy of our communication and also tap into the other, even more telling data.
But what to do? We can’t just opt-out. For most people, not using mobile phones is not an option. And frankly: I don’t want to quit my mobile. So how should we deal with it? Well, for people like me – white, privileged, supported by a legal system providing me civil rights protection, that is more discomfort than a real threat. But for everyone else, people that can not be confident in the system to protect them, the situation is truly grim.
First, we have to show people what the data does tell about them. We have to make people understand what is happening; because most people don’t. I am frequently baffled how naive even data experts often are.
Second, as Bruce Schneier argues, we have to get NSA and other governmental agencies to use their knowledge to protect us, to patch security breaches, rather then exploit these for spying.
Third, it is more important then ever, to work and fight for a just society with very general protection of not only civil but also human rights. Adelante!
Datarella now provides an API for our app ‘explore’, that allows every user to access the data collected and stored by the app.
An Application Programming Interface, in short API, is an interface for accessing software or databases externally. Web-APIs giving us access via the internet, have become the principle condition for most businesses in the web. Whenever we pay something online with our credit card, the shop system accesses our account via the API of the card issuing company. Ebay, Amazon, PayPal -they all provide us with their APIs to automatize their whole functionality to be included in our own website’s services. Most social networks offer APIs, too. Through these we can post automatic messages, analyze data about usage and reach, or control ad campaigns.
The ‘explore’ app was developed by Datarella to access the smartphones internal sensors (or probes), and to store the data. It is however not just about standard data like location, widely known because of Google Maps. ‘explore’ reads all movements in three dimensions via the gyroscope, accelleration, magnetic fields in the environment. Mobile network providers and Wifis in reception are also tracked. From these data we can learn many interesting things about ourself, our surroundings and environment, and about our behavior. To set the data in context, the API also gives out data from other users. For the sake of privacy and information self-determination, this is aggregated and averaged over several users, so that identification of a specific person is not possible.
With our API, Datarella commits to open data: We are convinced, that data has to be available for users.
➜ Here is our API’s documentation: explore.datarella.com/data_1.0.html
➜ Here the download-link for ‘explore’: play.google.com
We are excited to learn, what you will make from the data.
Around 2010, Lisa Gansky coined the term Sharing Economy, or Mesh companies, offering their customers efficient shared access to their products instead of selling their products to them. Recently, it’s being called Collaborative Consumption or Collaborative Economy. It’s all about finding ways to make better use of valuable resources that have remained unused. Convenient access is being made affordable to people who can’t afford different products, or simply don’t need to own those products since they would only use them infrequently.
Typical mesh businesses like AirBnB, LendingClub or Cookening, demonstrate the power of sharing in very different ways: AirBnB is on the way to pass Hilton as the world’s largest hotelier in 2015, that is 7 years after its inception. The US peer-to-peer lending company Lending Club has originated over 4 billion USD in loans – it was originally founded as a Facebook app in 2006. The typical Mesh business runs a stylish app with a high usability. It’s service is new, easy to use and affordable. But all that does not fully explain the tremendous speed they conquer one market after the other. Who is the driver behind the Sharing Economy and it’s success?
It’s the user.
It’s the user. The user offers and asks for private overnight stays on AirBnB, the user provides and lends money on LendingClub. Even with services like Zipcar, when the product is provided by a company, the user “uses” a product instead of buying and owning it. He has to rely on other users’ good maintenance of Zipcars, since if there were too many ‘abusers’ the company had to raise rates and the product would become unaffordable for most people. The same is true for AirBnB and others: users have to be sure that landlords don’t sell cubbyholes to them, whereas – vice versa – landlords have to trust their guests not to steal the TV or destroy the flat. So, the user has to use the service and she has to behave in an orderly manner – this is the foundation for a properly working and successful Sharing Economy.
Image: The Sharing Economy, Latitude
Now let’s adopt the principles of the Sharing Economy to the individual who shares her statuses with her social graph on Facebook, discusses the latest news on Twitter and shares her preferred fashion designs on Pinterest. She dos it because she wants to express herself and she wants to communicate with a wider circle of friends than she can meet in person. She communicates in both, synchronous and asynchronous ways. She has learnt that the more she adds to discussions, the more she gets in return. In Social Media, she experiences the Pay-it-Forward principle in action at its best.
Sharing Economy has attained full age in 2014
Let’s assume we can all agree on that: communication openly and actively, sharing ideas, opinions, homes, cars, money and much more with others is not an extravagant imagination of Utopia, Inc., but a multi-billion dollar business eclipsing traditional business models around the world. Furthermore, it’s not just a gigantic business but a sympathetic and friendly way of matching supply and demand of individuals. Who wouldn’t prefer an individually furnished private home over a standard hotel room?
If we agree on the power of sharing the above mentioned martial and immaterial goods, can we also agree on the power of sharing data? Our data? Our own body’s data? Can we agree on the tremendously positive and socially relevant effects of sharing the data we produce ourselves, day by day? If you own a smartphone (you most probably will), you produce about 20 MB of smartphone data (i.e. data racked with your smartphone’s sensors) each day. Perhaps you haven’t been aware of that fact, or you just didn’t know how relevant this data could be for yourself, and for your social graph, respectively. Do you know how much you move each day? The U.S. Surgeon General wants you to move at least 10,000 steps a day to prevent and decrease overweight and obesity. (It’s very easy to know your steps: just get yourself one of those fitness trackers.)
And, did you know that your Vitamin D level is one of the key drivers of your well-being? Most North Americans, North and Central Europeans suffer from a Vitamin D deficiency. Do you actually know your Vitamin D level? Do you know that you can find it out yourself?
Let’s get more complex regarding data: microbes in the human body are responsible for how we digest food and synthesize vitamins, our overall health and metabolic disorders. The aggregate of microbes is called microbiome. Do you have any idea about your individual microbiome? Do you know that you can find out about your microbiome yourself, by using a simple kit?
Sharing Data still in its infancy, but…
So far, we have talked about the relevance and value of our data for ourselves. But – weren’t you interested in preventing your first stroke because thousands of other men at your age have provided their heart and respiratory rates anonymously and based on the analysis of this data you had been warned early enough to take appropriate action?
Wouldn’t you agree that the Pay-it-Forward principle works perfectly in the field of personal body data? The difference to the AirBnB model is that you provide your body data anonymously. It will be aggregated and used in a way that nobody knows that’s you behind your data. Since data analysis and respective actions or recommendations rely on big data, it’s necessary that many people participate and share their own data.
… will become a Social Responsibility in 2017
Today, in late summer of 2014, many people are sceptical and hesitate to provide their data. We think that personal body data sharing will be regarded as quite normal within a few years. If this movement takes up the same speed as the Sharing (Goods) Economy, it will be accepted as “normal” within 2-3 years. We believe, that data sharing will become a social responsibility, comparable to fasten one’s seat belt or wearing a bike helmet. It probably won’t be called data sharing since this is a B2B term. There already is a very good term – the Quantified Self, or QS. The term itself does not include the sharing element. But for every active members of the QS movement sharing is a relevant part of the quantification process because the value of an individual’s data is even bigger if used for general purposes.
We regard data sharing – our Quantified Self – as one of the most important movements of modern times and we would love to know how you think about it: please comment, provide us with your feedback: do you already share your data? How do you do it? Or, are you still sceptical?
Feature Image: Max Gotzler of Biotrakr, presenting findings of a Testosterone study at #QSEU14
Sleep is one of the most interesting aspects of life: during sleep we don’t act consciously (apart from a few natural processes inside our body) and therefore some people try to minimize sleep to get most out of their lives. Others maximize their sleep: for them sleep simply is the greatest activity they could think of. The Quantified Self folks try to optimize their sleep; i.e. to maximize their sound sleep phases and minimize light sleep and times of being awake.
The guys from Jawbone looked at their UP band user’s sleep data and could provide us with this interesting global sleep pattern. Since Jawbone’s data are more detailed and accurate than the American Time Use Survey, this view on the different sleep patterns provides great insights in how inhabitants of cities behave, or how active a city is, seen as a whole. The average hours of sleep shown in the feature visual above do not include time awake in bed.
Science tells us that we should sleep between 7 to 8 hours per night and we should sleep during the same cycles in order to maximize recovery and relaxation from our daily routines. Now look at people living in Tokyo: with 5h 44min they sleep least, whereas Melburnians (yes, without the “o”) sleep most with 6h 58min – which is just the bottom end of the recommended length. Again Australians, this time the folks in Brisbane, go to bead earliest, at 10.57pm. The night-owls in Moscow hit the pillow almost 2 hours later, at 12:46am. But Muscovites (rise latest at 8:08am) sleep only 29 min less than the Brisbanian who raise at 6:29am.
Image: The author’s sleep pattern on August, 19 – provided by the Jawbone UP app
How about you? Do you know how long you sleep? Do you know about the quality of your sleep?
Above, you see a snapshot of my own sleep: On August, 19, I had 5h 22min of sound sleep and 1h 52min of light sleep – which is a pretty good ratio. I reached my goal of 7 hours of sleep as well, which is partially due to the fact that we are in the middle of school vacation and there is no need to mange kids in the mornings. So, don’t worry if your sleep looks less sound – I wake 1 time every 3-4 nights.
The most relevant criteria defining my personal sleep are
– duration
– cycle; i.e do I go to bed and rise at roughly the same times?
– alcohol input
– time without staring at a display before going to bed
– general family mood during the evening
I’d love to know about your experiences with your sleep. Do you track? What makes you sleep sound or light, short or long?
Today, we speak with Michael Reuter (KMR), Co-founder of Datarella, about the social relevance of the explore app guides.
Q
At Datarella, you offer different programs your users can participate in. Can you elaborate on the meaning behind these programs?
KMR
With our explore app, we provide a useful free tool for smartphone users to optimize their lives. There is a broad range of specific life situations in which the explore programs provide valuable and sustainable benefits. From lifestyle oriented programs as SMILE!, our guide to learn how to smile in 5 days, to specific health programs as our OsteoGuide which supports users suffering from Osteoporosis – we provide a broad range of programs. The most important aspect for Datarella is to always provide real benefits to our users: it’s not about technology, it’s about the social relevance of technology, its immediate impact on the user.
Q
Could you describe one of those programs and its impacts on your users in more detail?
KMR
Sure! Let’s take the OsteoGuide: in countries with populations with median ages of 45 and older, Osteoporosis has become a widespread disease. People suffer from Vitamin D shortage, move less and less during the day and, as a result, their bone structure becomes more fragile. If Osteoporosis is analyzed at an early stage it’s curable in most cases. To cure a patient from Osteoporosis you have to help her to regulate her Vitamin D level and to move more; i.e. to change her behavior: the patient should use the staircase instead of the escalator, or walk or go by bike instead of using the car or a taxi.
A change of human behavior is one of the toughest challenges you can think of. Ask yourself: how easy is it for you to quit smoking, stop taking the extra bar of chocolate, etc. The best method to support people in changing their behaviors is to provide them with instant feedback of their behavior and to give regular counsel in terms of notifications and recommendations. With the explore app and our programs, we cover these aspects perfectly. We accompany our users during a certain period of time and help them to change their behavior to the better, step by step, day by day. In case of the OsteoGuide, we cooperate with Prof. Dr. med. Reiner Bartl of the Bayerisches Osteoporosezentrum, an acknowleged expert in the field of Osteoporosis.
Q
That sounds fascinating: you say that people in need of medical care can get rid of their diseases by using the explore app?
KMR
To be very clear: the explore app cannot fully compensate a medical treatment. And Datarella is not a team of health professionals. We have to join forces with experts like Prof. Bartl to provide our share of a solution for a patient. But, in many cases, medication can only applied successfully if the patient herself contributes to her well-being. And, in most cases, this means that she has to change her behavior. We have string evidence that the explore app programs are perfect tools to achieve this goal.
Q
You mentioned that the explore programs are free. Where is your business model?
KMR
Yes, every smartphone user can download the explore app and apply for any of the explore programs. It’s free to participate on the basic program level which includes, tasks, notifications and recommendations during the complete program. If a user wants more, e.g. if she is looking for a personalized individual coaching, she would have to subscribe to the premium version of the corresponding program. With the premium version she would also get tasks, notifications and recommendations, but on an personalized level, customized to her individual needs. This coaching approach is mostly sought-after by users who must change their behavior in order to achieve a satisfying level of personal well-being. And if behavior change is a must, then you’ll look for the easiest way to reach your goal. The explore app programs fit very well into that requirement since the user will be coached in a soft, but equally demanding and rewarding way.
Q
Thank you very much for these insights!
