Some of you who know me well, know that I’ve been obsessing with an online service concept for about 5 years, trying to draw a structured map of the UK Technology industry. It’s been a concept I, along with some very good mates, have been exploring for such a long time we probably never knew when to stop. So, executive decision a few months ago – we need to launch it and get user feedback before we disappear further into our own little geeky world and get lost for eternity.
The journey started back in 2016, where over a Malbec fuelled “brain storming” session, I kept going on about how hard it is to know who does what in the Tech world. At the time I thought there were about 15,000 companies who provided IT Services in the UK, and wanted to build a database of all of them, structured with attributes, to make it easy for buyers for find the right supplier. We called it the Embedded Directory, and it didn’t work.
The concept was sound, but a “Directory” sounds too boring and easy for anyone to engage with. We encouraged about 100 companies to sign-up, had a handful of Buyer/Supplier match-ups and it dribbled along. Then during COVID I killed it, largely because it wasn’t quite what I was looking to achieve.
What it lacked was any form on “intelligence”. Creating a database of companies is actually pretty easy, but with little to no “analysis” it was just a directory. We needed a platform that covered the whole UK Tech industry, and automatically assessed each company for its core persona, ie who they were, what they did, how good they were etc. As a Procurement consultant, this is what I do. Assess a company for its suitability to engage on a given project, and award to the best fit based on attributes.
Relying on companies to sign-up was too laborious. For this to be holistic, we need to assess holistically. So, we set about getting a full list of Tech Companies in the UK – love you Companies House.
There are about 340,000 Tech Companies (defined by their SIC code) in the UK – and we are launching about 3,000 new ones per month – ranging from one-man bands (limited company contractors) to enormous long standing IT companies born decades ago. Companies House data shows us who they are, where they are located, and their ownership structure – but not much more. So how do you work out what they all do?
Of these 340,000 tech companies, the genuine businesses are likely to have a website which describes what they do. Finding the website address from a company name is not as easy as you think, but we found a way, and managed to identify about 90,000 websites associated with these tech companies. Bear in mind this has taken me 5 years. That’s a lot of googling.
Company’s websites are a treasure trove of information, or misinformation, based on your conspiratorial viewpoint. Analysing a website for keywords though, can unearth capabilities and focal points of what a company does. If a company mentions “AWS” on their website 10,000 times, they are probably into Amazon Web Services, Cloud, etc. Or they are massively into the American Welding Society, the Association of Women Surgeons etc.
So, we set about defining keywords that identified a company, and built an AI engine that looked for these words on a companies website, and built a series of attributes that could be ascertained:
Which technologies do they work with?
Whereabouts on the Stack do they operate (Applications, Infrastructure, Networking)?
What business model do they use (Developer, Service Provider, IT Support, etc)?
What key topics do they look at (Artificial Intelligence, DevOps, etc)?
What case studies do they have on their website?
Which industries do they focus on?
We aimed to build an engine that would read a website and build a persona of a company as if a human was doing the same. This is not trivial. But it works pretty well!
Final initial piece of the puzzle is a risk assessment. Primary risk, especially in todays economy, is financial and so we set about incorporating financial data from a credit agency to give a sense of credibility / size of the organisation, again acting as input to the online persona.
Online data sources for company research are amazing, and what you can find out is quite breathtaking. The issue is how do you consolidate this plethora of info into a useful summary rather than looking all over the internet (very time consuming).
Similarly, how do you ensure that a search service is not impacted by bias / agenda. Everyone is sick of the ability for big companies to pay SEO wizards, or pay per click ads on mainstream search engines, to skew the pitch and create a filter bubble of big companies only in the tech landscape.
We have a brilliant, diverse, and evolving technology market that should not be limited by new entrants ability to influence SEO / pay Adwords to get to the top of the pile. Buyers should be able to find a company based on their ability to delivery, not their ability to fund marketing.
So, “Embedded Intelligence” (name needs work…) tries to bring all this into a single platform. We are constantly scanning the online data sources to analyse as many of the 340,000 IT companies in the UK market as we can, to present a fact-based assessment (as fact based as we can make it with online data sources!) of a company’s suitability for a given project, or general capability.
It works as a search engine, where a search term is entered and results are ordered by “relevance” – how often a term is used on a website as a factor of the total amount of times it is used on all websites. That resulting list can then be filtered by attributes, such as location, accreditations, industry focus, credit rating, etc, to come down from a long list to a very tailored shortlist, giving buyers the ability to find a supplier based on who they are, not how much money they’ve spent on adwords. Suppliers can get a sense of what their online persona is, based on how a professional tech buyer would assess them, and ensure they are correctly represented. Or, a supplier could find another tech provider to collaborate with, do competitor analysis, assess M&A potential or pretty much anything required to find tech companies based on attributes.
As a bi-product, we have an aggregated view of the whole UK tech industry that draws some fascinating conclusions. Only 24% of all Directors of UK Tech Companies are women. 15% of all Directors at non-UK citizens. And we can drill down to region, individual postcode, sub-sector, company focus, age of company etc. It’s got great potential for journalistic content, policy influence, skills gap identification etc.
Data Quality is key, and we are acutely aware that this is a potential issue. We have safeguards in place to identify issues, but are constantly monitoring / adjusting parameters to ensure companies are accurately represents (based on the inputs).
We are at Minimum Viable Product (MVP) at the moment and are looking for early adopters for a heavily discounted subscription. We want feedback, engagement, use cases to develop the product in line with customer desire. Fancy getting involved? Get in touch we'll get you signed up!
Looking to the future, we are looking to incorporate as many public data sources as we can. Trustpilot, LinkedIn, cyber security assessment tools, Glassdoor, social media feeds, news sources, customer reviews etc. There is so much out there, and if you can consolidate and intelligently analyse you have the basis of a very high value map of the UK IT industry usable by Tech Buyers, Tech Suppliers and pretty much anyone working in the sector.
Can you see why I’m so obsessed about it? Cool huh?