These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.17 Jan 2018
I was integrating with the Clearbit API, doing some enrichment of the API providers I track on, and I found their API stack pretty interesting. I’m just using the enrichment API, which allows me to pass it a URL, and it gives me back a bunch of intelligence on the organization behind. I’ve added a bookmarklet to my browser, which allows me to push it, and the enriched data goes directly into my CRM system. Delivering what it the title says it does–enrichment.
Next up, I’m going to be using the Clearbit Discovery API to find some potentially new companies who are doing APIs in specific industries. As I head over the to the docs for the API, I notice the other three APIs, and I feel like they reflect the five stages of transition to the data intelligence dark side.
- Enrichment API - The Enrichment API lets you look up person and company data based on an email or domain. For example, you could retrieve a person’s name, location and social handles from an email. Or you could lookup a company’s location, headcount or logo based on their domain name.
- Discovery API - The Discovery API lets you search for companies via specific criteria. For example, you could search for all companies with a specific funding, that use a certain technology, or that are similar to your existing customers.
- Prospector API - The Prospector API lets you fetch contacts and emails associated with a company, employment role, seniority, and job title.
- Risk API - The Risk API takes an email and IP and calculates an associated risk score. This is especially useful for figuring out whether incoming signups to your service are spam or legitimate, or whether a payment has a high chargeback risk.
- Reveal API - Reveal API takes an IP address, and returns the company associated with that IP. This is especially useful for de-anonymizing traffic on your website, analytics, and customizing landing pages for specific company verticals.
Your journey to the dark side begins innocently enough. You just want to know more about a handful of companies, and the data provided is a real time saver! Then you begin discovering new things, finding some amazing new companies, products, services, and insights. You are addicted. You begin prospecting full time, and actively working to find your latest fix. Then you begin to get paranoid, worried you can’t trust anyone. I mean, if everyone is behaving like you, then you have to be on your guard. That visitor to your website might be your competitor, or worse! Who is it? I need to know everyone who comes to my site. Then in the darkest depths of your binges you are using the reveal API and surveilling all your users. You’ve crossed to the dark side. Your journey is complete.
Remember kids, this is all a very slippery slope. With great power comes great responsibility. One day you are a scrappy little startup, and the next your the fucking NSA. In all seriousness. I think their data intelligence stack is interesting. I do use the enrichment API, and will be using the discovery API. However, we do have to ask ourselves, do we want to be surveilling all our users and visitors. Do we want to be surveilled on every site we visit, and on every application we use? At some point we have to make sure and check how far towards the dark side we’ve gone, and ask ourselves, is this all really worth it?
P.S. This story reminds me I totally flaked on delivering a white paper to Clearbit on the topic of risk. Last year was difficult for me, and I got swamped….sorry guys. Maybe I’ll pick up the topic and send something your way. It is an interesting one, and I hope to have time at some point.
I was talking to my friends TC2027 Computer and Information Security class at Tec de Monterrey via a Google hangout today, and one of the questions I got was around managing API sessions using JWT, which was spawned from a story about security JWT. A student was curious about managing session across API consumption, while addressing securing concerns, making sure tokens aren’t abused, and there isn’t API consumption from 3rd parties who shouldn’t have access going unnoticed.
I feel like there are two important, and often competing interests occurring here. We want to secure our API resources, making sure data isn’t leaked, and prevent breaches. We want to make sure we know who is accessing resources, and develop a heightened awareness regarding who is accessing what, and how they are putting them to use. However, the more we march down the road of managing session, logging, analyzing, tracking, and securing our APIs, we are also simultaneously ramping up the surveillance of our platforms, and the web, mobile, network, and device clients who are putting our resources to use. Sure, we want to secure things, but we also want to think about the opportunity for abuse, as we are working to manage abuse on our platforms.
To answer the question around how to track sessions across API operations I recommended thinking about that identification layer, which includes JWT and OAuth, depending on the situation. After that you should be looking other dimensions for identifying session like IP address, timestamps, user agent, and any other identifying characteristics. An app or user token is much more about identification, than it ever provides actual security, and to truly identify a valid session you should have more than one dimension beyond that key to acknowledge valid sessions, as well as just session in general. Identifying what healthy sessions look like, as well as unhealthy, or unique sessions that might be out of the realm of normal operations.
To accomplish all of this, I recommend implementing a modern API management solution, but also pulling in logging from all other layers including DNS, web server, database, and any other system in the stack. To be able to truly identify healthy and unhealthy sessions you need visibility, and synchronicity across all logging layers of the API stack. Does the API management logs reflect DNS, and web server, etc. This is where access tiers, rate limits, and overall consumption awareness really comes in, and having the right tools to lock things down, freeze keys and tokens, as well as being able to identify what healthy API consumption looks like, providing a blueprint for what API sessions should, or shouldn’t be occurring.
At this point in the conversation I also like to point out that we should be stopping and considering at what point all of this API authentication, security, logging, analysis, and reporting and session management becomes surveillance. Are we seeking API security because it is what we need, or just because it is what we do. I know we are defensive about our resources, and we should be going the distance to keep data private and secure, but at some point by collecting more data, and establishing more logging streams, we actually begin to work against ourselves. I’m not saying it isn’t worth it in some cases, I am just saying that we should be questioning our own motivations, and the potential for introducing more abuse, as we police, surveil, and secure our APIs from abuse.
As technologists, we aren’t always the best at stepping back from our work, and making sure we aren’t introducing new problems alongside our solutions. This is why I have my API surveillance research, alongside my API authentication, security, logging, and other management research. We tend to get excited about, and hyper focused on the tech for tech’s sake. The irony of this situation is that we can also introduce exploitation and abuse around our practices for addressing exploitation and abuse around our APIs. Let’s definitely keep having conversations around how we authenticate, secure, and log to make sure things are locked down, but let’s also make sure we are having sensible discussions around how we are surveilling our API consumers, and end users along the way.
Profiling APIs always provides me with a nice bulleted list of what a company does or doesn't do. In my work as the API Evangelist, I can read marketing and communications to find out what a company does, but I find that profiling their APIs provides a more honest view of what is going on. The lack of a public API always sets the tone for how I view what a company is up to, but when there is a public API, profiling it always provides a nice distillation of what a company does, in a nice bulleted list I can share with my readers.
When I profile the APIs of companies like Amazon, Google, and Microsoft, I come out of it with a nice bulleted list of what is possible, but when I go even further, making sure each API profile has accompanying schema definitions, a nice list of what data company begins to emerge. When I profile an API using OpenAPI I always start by profiling the request layer of an API, the paths, parameters, and other elements. Next, I get to work describing the schema definitions of data used in these requests, as well as the structure of the responses--providing me with a nice bulleted list of the data that a company has.
You can see this in action with my Facebook API profiling work. There is a bulleted list of what is possible (API definition), as well as what data is sent, received, and stored (API schema). This work provides me with a nice look at the data Facebook gathers and stores about everyone. It is FAR from a complete picture of the data Facebook gathers, but it does provide us with a snapshot to consider, as well as a model we can ask Facebook to share more schema about the data points that they track. API and data specification formats like JSON Schema, and OpenAPI provides us with a toolbox to help us quantify and share the details of what data a company has, and what is possible when it comes to using this data in web, mobile, and device based applications.
I fully aware of the boldness of this statement, but I feel that ALL companies should have a public API definition, including a catalog of the schema for data in use. Ideally, this schema would employ commonly used standards like Schema.org, but just having a machine-readable catalog of the schema would go a long way to helping pull back the curtain of how companies are using our data. I am not asking for companies to make data public, I am asking for companies to make the schema for this data public, showing what they track and store about us. I know many people view this as intellectual property, but in an increasingly un/insecure online world of digital privacy, we are going to have to begin pulling back the curtain a little bit, otherwise, a rich environment for exploitation and abuse will continue to develop.
I love Evgeny Morozov's (@evgenymorozov) tweet defining the acronym SMART as Surveillance Marketed As Revolutionary Technology. It has provided me with a wealth of material for my alternate storytelling channels, and provides an excellent litmus test to apply to companies I come across during my monitoring of the API space.
As I'm reading do smart devices mean dumb security, out of Defcon this year, I'm reminded of his funny, yet also very troubling definition of SMART. I'm coming across an increasing number of connected devices who have incomplete API programs available. Meaning APIs are present, available on the open Internet, but required documentation, support, and other essential resources are missing--which like mobile, tends to often mean security and privacy considerations are incomplete as well.
This last week I talked about how venture capital investment can provide some incentives that are at odds with healthy, stable, consistent, and secure API operations. You see this play out with mobile devices, where a platform is so focused on the mobile app so heavily, they pretend the web APIs behind are invisible, which is also a practice I am seeing rapidly evolve with the Internet of Things (IoT).
Companies are racing to connect everyday objects to the Internet because they want to convince consumers to buy a new product, that will give them access to the valuable data that will be generated (a precedent set by the mobile evolution). In the race to create this new breed of products that consumers will want, and generate this new, highly valuable data, the willingness to secure these new data streams, and protect the safety and privacy of consumers is often very low on the list of priorities.
As stated in the BBC article out of Defcon, these devices will become a playground, of hackers, whatever their motivations might be. The average person will be unknowingly building out the Internet in this very unstable fashion, giving away their data, privacy, and of those around them. The greed behind the pushing of SMART objects into our personal and professional worlds will happily continue if they are given continued access to this extremely valuable data, and surveillance exhaust.
I'm not convinced that corporations, institutions, the government, or individuals will all be up to the task when it comes to securing all of this tech we are inviting into our worlds, not when there are so many badly behaved, poorly incentivized players willing to build this dystopian version of the Internet out. This will not play out well...
Need to go somewhere and you don't want law enforcement or even that nosey boss or wife of yours knowing about? Boy, do we have the person drone for you! The new facade edition of our microdrone can clone your iPhone, and can mimic all or part of your daily activities for you, while you are out doing what you need to get done.
All it takes is 30 days worth of log files on your cell phone, and your personal micro drone will perform any activity you choose. All you do is browse the available activities, and community behavior templates via your smartphone app, pick the chosen schedule and go. The drone will do the rest, even hovering outside your office window pretending you are at work.
While your micro-drone is acting on your behalf your cell phone will automatically go into an incognito mode, giving full control over the drone to broadcast your location, and even tweet, take photos and perform other activities as you need. Obviously, there are limitations, but to anyone tracking your location via your regular social channels, it will appear you are behaving as usual.
The new facade micro-drone is not just for eluding surveillance, it can also be used to broadcast new and interesting journeys, that maybe you don't have the time to take, using the community preprogrammed facades. There are trips to the park, zoo, beach, and many other activities you would like to be doing, but may not have the time or money to do so.
The new micro-drone facade is available on our website, or through the Amazon store--get yours now!
Two things Americans are suckers for are entertainment and convenience. We will give up almost anything if it makes our life easier, and keeps us entertained--no matter how simple that is. We love our movies, tv shows, and games, and we love everything to come to us from our shopping to our food, and our transportation.
This is where technology will continue to be employed in the name of surveillance--whether its corporate level surveillance or in the government sphere. This is where we will willfully accept surveillance into our lives, and allow for ourselves to be digitally pwned, allowing for us bit by bit to also be physically pwned--perpetually keeping us down.
Whenever possible let's pause the game, and think twice about signing up for that new delivery service, and consider what we are giving up in exchange for this entertainment and convenience. Are the tradeoffs worth it? Are we being distracted while our information is between taken, or the technology in our lives being compromised?
Let's not let a surveillance state creep in around us just because we couldn't go without for just a little while.
Utilizing Eagle Eye Network’s Managed PoE Switch with your Eagle Eye Networks system will provide an even greater level of manageability and more functionality than ever before, allowing for remote configuration and management. Now, authorized users will be able to power cycle an individual PoE camera or the entire switch remotely.
In 2020, why would you go with any of the mainstream home security providers? The RealityCom Reality Show Surveillance Package is tailored perfectly for the modern family. We do not just keep your family safe 247/7, we also help amplify the most important aspects of your life, and share with family and friends, and even the public.
Our critics call us a "modern surveillance apparatus", but these people live in the past. Privacy is a concept of the last century, and the modern family has embraced being not just a consumption family, but also contributing, and participating in the best reality programming out there. The most efficient, and cost effective way to keep your family safe and sounds, is through what we call "transparent living", where your home, possessions, and love ones are all monitored, and plugged into the RealityCom Security Network.
When our security production staff also finds an interesting scenario, our programming staff is notified, and we take the family moment, and streamed in near real-time to the audience of your choosing. Using the transparent living technology platform, you get to share with family, friends, and when we identify it as a quality moment of programming, we will pay you for the media and content--if your family becomes viral success, we pay you exponentially more, depending on the attention your family comands.
Not every family becomes a paid RealityCom Family, but you will not know if you have what it takes unless you get started with your Reality Show Surveillance Package. The best part is it is all free. We come out and install all the equipment, maintain all equipment, and store all the video, audio, and other content at no cost to you. You get modern home, auto, and work security for you and your family, for FREE! It doesn't get any better than this, sign up to day, so we can get started with your installation.
My job is to take the profiles generated by the lead analysts, and craft visual memes, that can be shared online. I have been developing my database of images for the last 4 years, allowing me to recall from a large imagebase, as well as prioritize images that have successfully met past objectives, or are used by my unit colleagues.
It is the lead analyst's job to know what each meme signifies, and how it translates into our targeting group speak -- I have never met anyone from that team. My job is to create compelling graphical memes the people want to share on Facebook, Instagram, Twitter, and other popular social networks. They have to be fresh, unique, and fit with the current flow of online memes, but speak to their intended audience--it takes a lot of work, to stay up to speed on what is the latest.
The surveillance meme generation unit has transformed how we track people at the NSA. We don't need people to say "I have guns in my house", we get them to share an "Obama is com'n for your guns" meme. We don't need you and your friends to admit they do drugs at the parties, we circulate relevant memes, and you tell us everything we need to know through your social sharing.
92% of the hate speech, religious propaganda, and drug culture memes are generated by our department. We churn out thousands of image and video driven messages a day, and aggregate, then index all derivatives, targeting all the concerning layers of society. Memes are the best way to profile a suspect in 2015. Period. Citizens do the work for us, all we have to do is set the tone for all of the conversations occurring online, for any given day.
I’m an advisor to the camera API platform, EverCam. I don’t advise the startup because I’m super excited about the opportunities for APIs for security cameras. I'm involved because I believe in the Evercam team, and I want to be aware of this fast growing aspect of the Internet of things and API economy. Security cameras are not going away, and I want to help lend some critical thought to how we use security cameras, and apply APIs to help introduce transparency and accountability into this easily abused layer of our society.
One of the things I learned from Evercam, is that in the UK you can request any photos of you taken on the vast closed circuit television, that is ubiquitous across the UK landscape. You can submit a request for a time, day and location and request any photo or video footage taken of you. Its kind of like a visual FOIA request for the surveillance layer of our society. This concept intrigued me, and I wanted to explore in relationship to other layers of convergence between the API economy and our increasingly digital society.
Imagine if there was FOIA process for data. I could submit a request to a single organization that would then make requests to leading technology, and big data companies, asking them for a copy of all data they possess about me, and disclose any partners that they have shared this data with. I know portions of this exist from companies like Acxiom, but I would like to see a more coherent, intra-company solution that could better serve individuals who wish to understand how companies are using their data.
A concept like FOIA for data across any company will not please corporate america, especially in a landscape where exploitation of users data is the predominant business model. However we are in the early years of the Internet, and things are very much the wild wild west, and it is only a matter of time before government regulations are needed to ensure the privacy of all citizens, and reduce exploitation and abuse by the bad apples.
This concept isn't far fetched. With modern, API driven systems, it is easy to track all of a users data, and where and how it is used across a company’s network. If all data access is required to occur via APIs, it will be easy to pull a history of which users were accessed, by which internal or external consumers. Each company could be required to have an API allowing a 3rd party auditor to pull data on behalf of users, allowing independent organizations to make FOIA style data requests across multiple companies on behalf of users.
I know that business owners will cry foul at such an idea, claiming it is just more unnecessary regulation that they will have to deal with, but we need a way of making all this more accountable. The API driven systems that would make this possible would also give companies all the other benefits APIs afford, in making company assets more accessible. APIs would allow companies to rapidly deploy web and mobile applications, while also providing assurances to every citizen that their privacy was being respected, and all of our vital personal information was not being exploited.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.