Big Infrastructure

It surprises me that for all the talk of big data, and the huge investments rushing to capture niches in the big data space, so little attention is given to the infrastructure that will need to move that data.

Today's internet is a mostly-wired infrastructure paid for by the owners of the end-point systems, be those consumers who have a wireless router in their living room, or consumers with mobile devices that connect wirelessly to the local cell phone operator.

It isn't at all clear to me that either model can work for tomorrow's world in which the Internet of Things will be capturing much of the data and taking many kinds of actions: a world of small devices that will often be installed in the relative "wilderness" of our cities, highways, buildings and other spaces. 

Consider a small sensor playing some sort of role in a smart building: perhaps it monitors temperature and humidity in a particular room.  This little device will need:

• A stable source of electrical power.  You may be thinking "batteries" and sure, that can work, but this assumes a consumer willing to replace those batteries once a year or so.  If our device wants to generate its own power, that could be possible.  I've seen work on harvesting ambient EM power from background signals, and of course our unit could also have some form of optical cell to convert light to electricity.  But until that question is solved, the IoT world will be a world of things you plug in, which already tells you something about "form factors" and roles.
• Some idea of where it is located.  If the sensor in the room is telling the house whether or not to run the A/C for that area, or the dehumidifier, clearly that sensor has to know which room it is located in.  Perhaps even which way it is pointing: if you want the lighting for guests on the couch to be just right for reading, or for whatever else they may be up to, you need to be pretty smart about the room.  A smart traffic light wants to know which intersection it is at, on which corner, and which way it is oriented relative to the streets.  Traffic sensors will want to know these things too, even when located in the middle of a block or on a pole high above a freeway.  A smart printer wants to know who is allowed to use it, and your phone wants to know which of the printers it can sense belong to your home, or to your friend's home if you happen to need to print something while hanging out elsewhere.
• Some idea of role.  Many sensors will be created from the same basic elements (sensors that can detect light, or take photos or video, or track moving things in the vicinity, or its own motion, or humidity, or sound, etc).  These general purpose units will be cheap to manufacture in huge volume, but then in general will need to be programmed to do some specific thing that may use just parts of the functionality available.
• Who's in charge.  Small devices will generally operate under control of more centralized ones, which in turn might talk to still large-scale controllers.  When you turn the device on, how can it figure this out?
• How to network back to the boss.  Even a tiny device may have many options available: Bluetooth, various WiFi options, cellular telephony options, you name it.  Which should it use?

Then we run into questions of the network side of the Internet of Things.  Will this really be the Internet, or will the IoT demand some other style of networking? 

For some uses, today's network probably can get us there.  For example, I think one of the biggest things that will be coming soon is delivery of TV-style live content, including various forms of group video conferencing, or live sports events.  Those applications center on moving bytes, and our Internet service providers are pretty motivated to enhance their networks and then sell us more and better bandwidth.  The technology to build data centers for replicating data rapidly inside the data center is in hand (Derecho!), and the existing last-hop options are probably fast enough, so we can get that video stream to the end user without a massive upgrade: it really could be done.  But TV turns out to be a fairly easy case.

Networking for little devices isn't at all clear. Here we run into an issue like the gigabit network last leg: something enormous needs to be built, yet it currently isn't monetized, even to a limited degree.

A student I've worked with is convinced that standard networking isn't really suitable in many of the environments that the Internet of Things will run in: he points out that the people who service traffic signals aren't networking experts, and yet when traffic lights evolve into smart signaling units, those same people will probably be the ones in charge of replacing broken devices.  We'll need a very different model for how the broken system reports its faults, prioritizes repairs, and how it communicates all this to the infrastructure owner and the repair crew.  Since failures and environmental challenges (like ice, snow, humidity, physical damage...) are going to be common in many settings, and some devices may be in fairly remote and isolated settings, he believes that repair won't be instantaneous either: we'll want networking redundancy, using ideas like duplicate links, RAID-style data coding across the various available network pathways, and fault-tolerant routing.

Security will be an issue too.  It is nice to imagine the benefits of moving towards a world of smart factories or smart transportation, but we'll want this world to have much stronger security than today's Internet (which probably wants much stronger security than it actually possesses!)   But quite likely the sensors in the Internet of Things will need built-in security keys, and the architecture will be such that a given device will only accept instructions from properly authenticated, properly authorized control units. We'll want to encrypt the data sent and received, both to protect against unauthorized spying, and to ensure that the received commands really originated at a trusted source.

And then all of this will need to connect to the cloud, since the whole point of the Internet of Things will be that by using "big data" tools to learn about environments, we can often optimize them in useful ways: to waste less power, for example, or personalize the environment so that a room is a bit cool if I'm it, but a bit warmer when my sister comes for a visit (she's used to a much warmer climate and finds the places I live most of the year frigid).  We might even imagine personalized advice: today is February 7 and I was swimming in the Mediterranean sea off Tel Aviv.  For me, the advice would be that the water was chilly but quite swimmable.  For my sister, same beach, and the message would be "deadly cold, don't even think about it!"  The cloud will be the entity in this picture that builds up the personalization story: Ken likes it kind of chilly; his sister prefers it a bit on the warm side.  So the Internet of Things will feed data to a cloud that in turn takes actions through both small actuators (like the traffic signal) but also through our mobile devices.

We're going to need a new technology for building and operating these systems: a new kind of "Big Infrastructure", more or less the physical counterpart of today's "Big Data" analytics and tools.  But surprisingly little work seems to be underway on creating that technology, or deploying it.

Here's the very biggest puzzle of all: who is going to pay?  In fact the real answer is obvious: in the end the consumer will pay, either directly in the sense of finding the technology so valuable that he or she voluntarily buys it, or indirectly, through municipal taxes and fees that pay for the smart traffic lights, or even by accepting personalized advertising.  Money is invariably the key: spend enough, and most problems can be solved, and the money tends to be where the majority of the people are to be found.

Viewed this way, it becomes clear why the Internet of Things revolution is stalled, at least for the time being.  The value of doing an Internet of Things is pretty clear.  Once it exists, it will be an immense value generator, so if you could solve that chicken-and-egg puzzle, you'll enable a very, very lucrative new world.  But until it exists, we have a version of the so-called last-mile problem: it is very easy to get gigabit networking to within a mile of everyone, maybe even to within 100 meters.  Then the costs simply explore for that last leg, and while we would see a tremendous burst of innovation if we had gigabit networking into our homes, the cost of bridging that last leg is a tremendous barrier to starting to reap the benefits of the faster technology.  Thus nobody actually has gigabit networking now, so the applications that would be enabled by that sort of development aren't yet being created: if you did so, nobody would buy them, because few people have a fast connection.

Earlier I said that live TV is coming, but that's because the bandwidth seems to be within the range that the existing infrastructure can manage.  With a gigabit technology we could do far more, except that as far as I know, nobody has yet figured how to upgrade that last leg to support it.  And the small devices version of the problem has this even greater puzzle of lacking an existing paying customer.

Where will the big infrastructure come from?  If you figure this out, let me know: I'll want to be an early investor.