Thank you for offering to sell me public data, but I think I'll pass.

A company I like approached us recently, offering to sell us a "schools collaboration portal". The sales pitch was something like:

We can create a bespoke collaboration portal for you so you can compare your schools to any schools nationally. You can find similar schools and then contact them so you can share best practice. Imagine how amazing that would be.

After some clarification questions, I think their pitch could be better summarised as:

Please buy free, public data from us. It's worth paying for because we've put it all in one place with a pretty CRM front end. 

Hmm.

To be fair, it's not the worst idea in the world. Just because data is public, I get that software to analyse it might not be. And the people that pitched the idea are impressive and smart. Also, they aren't the only ones thinking about this, because a few minutes of googling (or "binging", to quote our Microsoft relationship manager), led me to stumble across schooldata.org.uk, who seem to offer a similar product with a freemium model. I've signed up and am looking forward to having play once my account is validated.

But clearly, free would be even better than freemium. So I'm dearly hoping that the rumours are true and that the fragmented mess that is Edubase, the performance tables, Ofsted reports, Ofsted dashboards, and Idaci data will be combined into a single portal very soon. That would immediately raise the bar for what is worth paying for. (I'm not saying these are bad initiatives in themselves, by the way. I like the Ofsted dashboards in particular. They're simple and graphical, which are both good qualities for data dashboards to demonstrate. But they're only a small part of the picture.)

And I still think my dream product is much bigger than whatever the DfE or gov.uk (or schooldata.org.uk) is likely to do. Imagine if someone cast the net wider and also built in population trend data, or the pupil intake ward data that's in the London Schools Atlas. Basically, any publicly available data that helps inform our view of what's going on with schools nationally. It would be a hugely beneficial open source project. Anyone up for it?

The technical challenge is not overwhelming. In the absence of a combined government portal, a few of us at ARK went ahead and created a National Schools Database in Salesforce. We've uploaded all the data sources I listed a couple of paragraphs ago, plus a few other bits and bobs. It's still quite homemade, but it's useful nonetheless, because we now have one place where we can go to answer any question about key schools data. And the fact that Salesforce is a CRM is a bonus, because we can add comments, notes and additional data in custom fields whenever we like. Perhaps we should have done it in CiviCRM - then we could have produced a fully "pre-loaded" version for others to use for free?

Anyway, geekery aside, my main point is that information wants to be free. And that is particularly true of public data. So thanks for offering to sell it to me... but I think I'll pass.

Salesforce-style platforms are the future of edtech

In my previous blog I mentioned my recent talk at BESA's edtech special interest group. A central theme of my presentation was that schools need cloud-based platforms, not standalone desktop products.

I worry that people like me are prone to saying "we should move towards the cloud" without being clear about what we mean. Jose Diaz at ARK worries about this too. I'm paraphrasing, but I think his (very sensible) challenge to me was something like: "do you mean we should move towards browser-based software instead of desktop products (which could be achieved with local hosting) or do you have a theory of why the cloud is better than local hosting?"

I assume the cloud does offer benefits compared to local hosting, such as simpler backing up, cheaper and more reliable hosting, and easier upgrades. But I imagine those benefits could be mitigated in other ways. To be honest, the infrastructure aspect isn't my area of expertise.

So I'm going to stop saying "we should move towards the cloud", and start saying "we should move towards cloud-based platforms". And I think I should focus on the "platform" bit of that sentence in particular, because we need great edtech firms to be building apps that bolt on to platforms with common standards. We really, really don't need endless standalone products that do not integrate with each other.

I think Salesforce is the best example of what a winning platform should look like. It started out life as a tool for managing customer interactions, but that isn't really what it's about anymore. The way I think of it is as an endlessly configurable database, linked to a simple user interface, with great integration tools and a built-in app store. The technical term for this kind of approach is Platform as a Service (PaaS). The Salesforce website gives a fuller description of what this really means. It also offers a nice explanation of how this links to Software as a Service (SaaS).

Definitions aside, the crucial thing about Salesforce for me is that you can make most of the functionality changes you'll ever need through configuration clicks, not coding. When you need to make bigger changes, you can usually find a plug-in app to do the job. Either way, if you can define the business logic, you can usually mould Salesforce to make it fit.

Platforms like Salesforce are different from the app stores that link to operating systems, such as iTunes (Mac OS) or Google Play (Android). App stores give you super-simple integration with your device. Platforms give you close integration between apps and your core business processes.

Now, imagine if something like Salesforce existed in UK schools. Users wouldn't have to familiarise themselves with a new interface every time they buy a new product - they'd just plug it into their platform and get started. Curriculum software would integrate with a school's standard approach to testing and data analysis. Cost would also come down, in the same way that mobile apps have radically disrupted the price of consumer software. It would be awesome.

It would be wrong to imply that nobody has thought of doing this already - in fact loads of people are trying to build an app ecosystem. Frog has the Frogstore. RM Unify is an app store for education. Moodle relies on its plugins directory. Sharepoint links to the Office Store. So the problem isn't aspiration. The problem is that hardly anyone in education is using platforms in this way. I'm also not yet convinced that any of these platforms can match Salesforce for configurability and simplicity of integration. (I can't find any data on the current market share or usage levels of these platforms and app stores - if anyone knows more, I'd love to hear from you.)

I asked the 13 UK edtech companies at the BESA session which of them were integrating with a learning platform right now. Most had not even tried. A few had experimented with Frog or Moodle integration, but had seen minimal customer uptake. One told me they were considering RM Unify, but they hadn't built anything yet. So basically, they're all still selling standalone products. I don't blame them, because there's no money in creating a great app that nobody buys. But this has to change.

We really need two or three platforms to take off. We need people like Renaissance Learning to feel confident that if they make products like Accelerated Reader available on platforms we'll actually buy them. And in an ideal world the winning platforms would extend beyond learning. They'd also make it easy for productivity, finance, HR, MIS and other school systems to plug in.

At ARK we're currently reviewing our entire approach to learning platforms. The route we take will be the one which we think leads us, and all UK schools, to the most Salesforce-style vision of the future.

I do not believe you when you say you know "that company"

I spoke today at the snappily named BESA SATSIG. It's a UK edtech special interest group, and also a very nice bunch of people who offer excellent sandwiches and wine at lunchtime in exchange for a lively and mutually interesting chat. I like groups like this.

However, as they each introduced themselves, it brought home to me just how much I still have to learn about UK edtech. These are solid, decent-sized businesses, yet of the thirteen present, I knew nothing at all about six of them, a little bit about five, and a good amount about just two.

Now, there are two conclusions you could reasonably draw from this. The first is that I am in no way qualified to call myself an expert in edtech (and you'd be right). The second, and the one I want to focus on in this post, is that people's knowledge of their own sector is often scarily limited. The problem is particularly acute when it comes to something as dizzyingly fast-moving as education technology.

Here is a summary of what I often hear people say about companies, and what I think they mean when they say it:

• Oh yes, I think I know the name. Oh no, I do not know the name, but you said the name so enthusiastically I really feel like I should know the name. I hope I remember to google the name later. What was the name again?
• Sure, I know those guys. I walked past their stand at a trade fair. I vaguely remember their logo. There were no free pens, so I didn't linger. With hindsight, perhaps I should have lingered. 
• What they're doing is really exciting. I don't really know whether what they are doing is exciting, but I'm chuffed that you mentioned a company whose business model I can just about explain.

And so on. Anyway, my intention is not to poke fun at the ill-informed. I know I've bluffed before - it's a natural human instinct. My point is that the sector is so young, and evolving so fast, that we should all feel confident enough to admit what we don't know. 

Also, the knowledge gap is inevitable just from the sheer number of players in the sector. There are 822 products currently listed on the excellent EdSurge Edtech index. It's US focussed, so I have some justification for not knowing them all. But given that I work with education data, I feel like I should at least know a bunch of the 26 US data systems they list. So I tested myself. This is how I did:

• Know nothing: 22
• Can tell you what it does: 2 (Clever, Learnsprout)
• Have actually been in touch with them: 2 (Schoolzilla, Ed-Fi)

Not great, is it. If you'd have asked me before I undertook the exercise, I might have claimed that I knew a bit about the US education data landscape. In fact, it turns out what I mean is that I read Edsurge and Techcrunch sporadically, I've read Driven By Data, I've spoken to three helpful edtech investors, and been in touch with two actual US data companies. Bully for me. That's tip-of-the-iceberg stuff. So: time to start googling the other 22...

The structure of the education sector makes data innovation really hard

I've never been a teacher and I'm not a statistician. So there are loads of people who you should ask about school data before coming to me. People like Daisy Christodoulou, ARK Schools' R&D Manager, for example. My job involves improving the way we use data and systems, but I defer to others on the educational philosophy which underpins our approach.

Even from a technical perspective, I am not anywhere close to being an expert. Instead, I would describe myself as at the "repeat-what-clever-people-tell-you" stage. So here's what clever people have been telling me over the past months:

1. People are confused about what school systems do (part 1). The core administrative system in UK schools is usually referred to as the Management Information System (MIS), but I don't think this is a great description. To me a MIS is a system that lets you analyse data for management purposes. School "MIS", on the other hand, are always used for data collection, and only sometimes for data analysis. So it would be more accurate to call it an Information System.  
2. The "MIS" market is not structured to foster innovation. 83% of schools in England use Capita's desktop-based SIMS product (this helpful Edugeek post has the full breakdown). Cloud-based systems such as the promising, primary-focused Scholarpack are barely 1% of the market. Capita are not exactly hurrying towards the cloud. Maybe this is because they are happy with the way the market is working for them. Maybe I'd feel the same if 17,912 schools were paying me to sell them a legacy product.
3. Gathering data is hard... Plenty of schools do have an assessment model and so generate data, but it can be a huge endeavour to get the data of a quality that stands up to proper analysis. For example, are you holding common tests across age groups? Are you moderating teachers' marking? Do you know how to tie your progress measure to curriculum?  
4. ... and analysing it can be even harder. Not all "MIS" have good analytics, and some are really bad at it. So many schools choose to handle analytics outside their "MIS" by using products like Target Tracker or Go4Schools. That's fine, but it can feel like double-paying - particularly when the core product is called a Management Information System. 
5. People are confused about what school systems do (part 2). Lots of schools have something they call a Virtual Learning Environment (VLE). Others call it a Learning Management System (LMS). A few call it a Learning Platform (my preferred term). Whichever way you badge it, I'm not convinced that many schools are using them for learning. A more accurate description would be "fancy shared drive", or "intranet". In other words, I've mostly seen them used for passing around documents (eg handing in homework), and for school administration (eg booking a room). These are useful functions, but they're not the same as online learning and assessment.
6. No-one in the UK is blending data well. Even schools that do analytics well are unlikely to have managed to blend data from multiple source systems. The main problem is that analytics is usually tied to a single source system. For example, you might use SIMS Discover to analyse your SIMS data. That will do a job for you, but what if you want to look at SIMS data alongside data from your learning platform, or your HR system, or your finance system? In a big data world, it's usually preferable to handle analytics outside of your source systems, for example by blending data in a data warehouse and then attaching that to your analytics tool. That's what we're working on at ARK Schools right now, taking inspiration from the hugely impressive Schoolzilla in California.

This brings me to my central point: these problems persist because the structure of the education sector makes data innovation really hard.

Things like point 6 above are mindbendingly complicated, and you need specialist resources and lots of time to get a solution in place. In most multi-billion-pound sectors, you'd expect to find a few big fish with huge technology budgets who can commission clever solutions. Smaller fish would then benefit as the solutions trickle down to them over time.

But schools can't do that, because there are no big fish. There were 24,327 schools in the January 2013 census, and the largest mainstream school was Nottingham Academy, with 2,543 pupils. I'm guessing that gives them a budget of £12-13m. After teachers have been paid, that sort of scale doesn't leave much room for big data innovation.

Local authorities arguably have the have the scale to help, and many do employ smart data people, but the number of schools under local authority control is declining as the academies sector grows. This inevitably leads to shrinking budgets, and shrinking budgets are not the best place to look for whizzy innovation.

Academy groups should help to fill the gap, but I wouldn't overstate the extent to which this is happening yet. The sector is very young, and growing rapidly, and as a result many groups are struggling to cope with even the most basic systems challenges.

ARK Schools have done a lot with data - I'm sure I'll write about the awesome reporting tools built by Jose Diaz in future posts - and I'm excited about where we're going. But I'm still getting my head round the fact that we can't really improve by copying best practice, because best practice doesn't exist yet.