Episode Transcript
[00:00:00] Guest: Everyone talks about wanting to have a data culture, which is great, and I think it's the right idea. But a culture means that we all have a certain set of shared abilities and ability to investigate and critique and pursue our interests and, you know, challenge each other in that common space. I just don't think that you can call yourself a data culture if only four people have access to the data.
[00:00:31] Host: Welcome to the EdTech Connect podcast, your source for exploring the cutting edge world of educational technology. I'm your host, Jeff Dillon, and I'm excited to bring you insights and inspiration from the brightest minds and innovators shaping the future of education. We'll dive into conversations with leading experts, educators and solution providers who are transforming the learning landscape. Be sure to subscribe and leave a review on your favorite podcast platform so you don't miss an episode. So sit back, relax, and let's dive in.
Jeff Rubinstein is a leader in educational technology with a track record that spans Google, Kaltura2u and Wimba. He's built and led product teams focused on learning management systems, learning analytics, collaboration tools, and AI powered solutions that actually work for educators, not just in theory. Today, Jeff is the Chief Product Officer at Dewey, a company using AI to turn education data into actionable insights for teachers and institutions. He's also a frequent speaker, writer, and self described amateur ed tech historian who brings both curiosity and a critical eye to how we use technology in schools. Jeff's unique mix of startup grit and big tech vision helps him cut through buzzwords and focus on what matters most. Helping students learn and supporting educators with tools that make their lives easier.
Welcome to the show, Jeff.
[00:02:07] Guest: Thank you. And by the way, congrats on having a lovely first name.
[00:02:12] Host: Yeah, it's like, it's a very generational name. I feel like there's not a lot of baby Jeffs coming in the world now. There's not a lot of grandpa Jeffs. It's kind of like this middle aged guy name, but yeah, well, and also.
[00:02:22] Guest: If I had been female, I would have been Jennifer. So, you know, it's, it's that era.
[00:02:28] Host: I gotta ask my parents. That one would have been. So You've worked at Google 2U, Kaltura, other companies.
I want to hear like your journey. Tell us how you got to where you are at. Dewey. And did I say it right? Did I say the name right? Dewey.
[00:02:43] Guest: Dewey, yes. So it's named for John Dewey, the American philosopher and theorist of education. We of course spell rdue d o dash o w I I because-e dash w dash e dash y dot com was taken. So that's how we ended up with the spelling of our company name.
[00:03:02] Host: Right.
[00:03:03] Guest: So I've been kicking around EdTech about 30 years in a variety of startups in and around the enterprise systems that help teaching and learning in particular.
And one of my side interests has always been student data and what we can do with student data to help promote better learning. And I've done that through all of my companies and also alongside and in participation with the group known as One EdTech, formerly known as IMS Global.
I can proudly say there's a teeny tiny bit of my DNA in some of those standards like LTI and caliber and that sort of thing. That's one of the things that I've enjoyed doing over my entire career. And one of those efforts has always been trying to solve the student data problem. Given that student data is smeared across a whole number of systems, which kind of makes sense given the genealogy of most of these systems. They each arose at a different point in time and they each arose to solve a very specific problem. So they're, they're more like a number of point systems that have arisen over the years. First you had the sis, which didn't have any student face to it. It was just the back end system that knew your demographic information and your course rostering. Then in the early aughts became the era of the lms, which was the sort of collaboration space where students and teachers met and could share documents like the syllabus and some course content. And there was a bit of data in that system. But very quickly a lot of student activity moved out into these peripheral systems, the ones that tend to be connected via lti, things like digital products from the publishers like, you know, McGraw Hill and Cengage and Pearson, third party specialist tools like video platforms or proctoring tools. You know, every school has an ecosystem of those. And because those arose as fully third party systems which you launch into from the lms, all those systems have a piece of the student data as well.
And so for years we've been trying to figure out, okay, I mean, there's some information in each of those systems that's useful. The SIS has demographics, the LMS has activities in the LMS like document views and some quizzing. These third party systems have information like the quizzes you might take in a virtual science platform or a virtual chemistry platform. What you do in a virtual classroom session, like how often you raise your hand, how often you speak, and wouldn't it be great if we could kind of get all this data together and be able to use it to help students learn better. So that, that was some of the work that, you know, lots of people, not just me, have been doing over the years.
And so now we're at the space where in some cases the data is getting put together, which is great. And more and more universities and K12 schools are getting on board with the idea of some kind of data lake where this data comes together from all those systems and then we try to structure it into something that's meaningful.
And the thing that I started doing at Google with the beginnings of AI was we were starting to. Of the many things you can do with AI, and AI covers a pretty wide swath, one of the things it can start to do is start to do basic data science and data analysis. Because where in the old days you had to have some kind of data architect understand, this column over here in this database is the same as this column over here in this database or transformed in this way, AI can start to automate that. And so we started doing that at Google.
And that brought me now to finally come back to answer your question to Dewey, where that's what we do exclusively, which is we try to use AI to automate the data science process for schools in particular for the following reason. Let's say you are a school and you are somewhere along the journey of getting this data from all these sources into one place, in some sense of one place, you're then left with, okay, how do you actually get insights from that information?
And most schools just haven't gotten far enough down the path to think beyond the obvious but not entirely satisfactory answer of, well, I'll hire some data analysts and they'll use Tableau or Power BI or whatever, because while that's not a bad answer, and that will in fact work to a certain degree, there's one big problem, and that's these humans, these data scientists can only do so much so fast. And they're going to be tasked, of course, with the most important, expensive and strategic kinds of reporting that the school is going to need.
Which means that in reality, here's what's going to happen. For years, the way that anybody else in the org has gotten data is to put in a ticket to it and say, please, sirs and madams, may I have some data, please? And may I have some more?
[00:08:51] Host: Yeah.
[00:08:52] Guest: Then often the answer comes back, we'll get to it never.
And that's just not a great answer. So the purpose of doing is, can we actually use AI to create a virtual data scientist or virtual data analyst that will allow anybody, without needing to go to it, to please get data or please give me a dashboard or please update this dashboard to give them the data and the insights they want so they can take action right now and improve the life of a student.
[00:09:32] Host: When you say we don't need it, do you mean after implementation, like isn't there some sort of API connections or are you crawling data or how minimal is that it lift?
[00:09:45] Guest: Really great question. And certainly it is needed to make sure that we have the data in the first place.
And that can be done a few different ways depending on where the org is in their journey of starting to aggregate this data into one place. So we do need it to help with that piece of it. But once we have that, we can then democratize access to that data and give access to admissions, student support, career services. Really anybody who is authorized to have access to be able to get the information they need to do their jobs.
[00:10:24] Host: So help me understand.
I think it was about five years ago I ran to a company called Invoke Learning and predictive analytics was the big thing right back until it's still pretty incredible. It just kind of got surpassed by all this AI stuff. The more we can do now, right? So if we analyze all this data, we can predict like, yeah, you did not spend X amount of time two weeks in the semester. If students aren't clicking X amount of times or logging X amount of times, we kind of know 80% of them are going to fail. I mean there's these stats that are like, you know, we can predict that. But also they took it the step further where it was. We know all the zip codes of these students right from the sis and we actually they were connecting to external data sources too to say, hey, there was a power outage. This is a low income neighborhood. They couldn't turn in their assignments. Like they can see patterns of like why students that live in certain areas couldn't get things done. So what level can we really predict? Are we in the predictive analytics arena too?
[00:11:23] Guest: At Dewey we are and we're doing some projects right now exactly along that predictive line of thinking. Given data from the SIS and the LMS and the associated systems. I think where we add an extra twist is that we can do that for non technical users and without even needing necessarily somebody to set up a lot of that upfront. I'll put it this way, you know, lots of tools out there are very powerful and they can create dashboards which can then be published. And that's the current sort of state of the art of disseminating information. That doesn't give people the agency to do what you would want to do if you had a really strong data culture. And let me explain what I mean by that. Everyone talks about wanting to have a data culture, which is great, and I think it's the right idea. But a culture means that we all have a certain set of shared abilities and ability to investigate and critique and pursue our interests and, you know, challenge each other in that common space. I just don't think that you can call yourself a data culture if only four people have access to the data.
[00:12:39] Host: Right, right.
[00:12:40] Guest: And so the question for us is, how can we extend this ability to have these shared objects in a shared space and allow people to investigate things without having to go through those four people or five people who have the tooling and the experience? Again, we're not in competition with power, bi and tableau, and there's always a place for that. But we want to open up the space and we want to enlarge the space here for everyone. And that means not only having the analytic capabilities, but being able to allow non technical people to use them. And so that means not only if you sort of look at the hierarchy of data value, at the bottom you have aggregation, you have to kind of get it all in one place to use it. On top of that, there's some basic work to get it ordered and structured. And you know, what you would call data marts, you know, in a database type vocabulary and have some tooling on top of that where experts can use it. Right. And that's great. That is a lot of value, don't get me wrong. But then if you want to go further and extend that value to other people, you need to have built in a lot of semantic understanding of that data so that you know what it means and you know what people are asking for when they ask for it. Like if somebody asks for students who are chronically absent. Right. That phrase is nowhere defined in your data. Right. There has to be an interpretation there. Using techniques like a knowledge graph, you know, which is common in AI, that says, oh, okay, when a human says chronically absent, that means having missed 10% of available school days. Right. Or show me my enrollment in liberal arts.
[00:14:34] Host: Right.
[00:14:35] Guest: Liberal arts is not a data point anywhere in your database.
[00:14:39] Host: Right.
[00:14:39] Guest: That, that means anthropology and sociology and economics is a human concept. And so naturally, when a human asks for data, they're going to ask for it in human language.
And you have to have a really good semantic understanding of what that means applied to your data. Now this is something a data analyst will have in his or her head, which is why most data interactions are a dialogue with a data analyst who says, what are you looking for?
Okay, I think that means this. Test my assumption. Okay, I don't have that data point, but I have this, which is a proxy for it. You know, if I give you these and then you have a, you have a back and forth that assumes you happen to have a data analyst available.
And so what we're doing with Dewey is we're providing that dialogue that back and forth with AI so that anybody can get to the point of doing predictive analytics or enrollment or student success or whatever it is that moves their needle.
[00:15:44] Host: Yeah, I love the message of democratizing access to this information. It's similar to our mission here too at EdTechConnect. So I'm thinking about all these systems you mentioned. The early on systems, everyone had an LMS and an sas. Now there's so many more. There's video content management systems and everyone's kind of racing to integrate AI in some of the ways you're talking about. Everyone kind of knows that's where they need to go. And so we have all these logins and some schools have done a good job about single sign on and, you know, how do we seamlessly make this user experience manageable or even elegant for these students?
Explain to me how Dewey, you know, is it a standalone system where you're going to have a Dewey login to the Dewey Dashboard? Or, you know, I mean, these SIS lmss have this huge kind of head start and they have all the data already. And is it something where you're going to see in your LMS like it's within the lms, or is it both? Or it kind of goes into your business model too, I guess is how does that.
[00:16:43] Guest: Sure, we do it both ways, frankly. One way we do it currently is that we do power other technologies with our API and our services. So, for instance, this is not a secret, but it's not particularly talked about widely.
The Canvas LMS has a feature set called Intelligent Insights and a part of that offer is Dewey technology that's embedded into Canvas that certain levels of Canvas admins can have access to. We also do standalone work with a lot of both higher ED and K12 clients. And in that case Dewey is a standalone app, although in other cases we're now working at integrating it, say into the frame of salesforce.com, which is used as a case management tool in many cases for student services. And they are using Dewey technology to say, hey, show me my students who may be at risk or haven't been as engaged as they might have been. Where should I spend my time? And that will appear to them inside their salesforce.com frame as a part of just their console that they know as their student success tool.
[00:17:54] Host: Yeah, that seems like the way you have to go. Right? Like really partnerships with these big companies is going to can really accelerate your adoption in this competitive market. And a Dewey dashboard would be great too.
[00:18:08] Guest: Indeed.
[00:18:09] Host: So can you tell us some success stories or practical examples of what this means for educators?
[00:18:17] Guest: I'll give you a couple of, I think key things that are really useful. Number one is at the student success use case. You know, right now student success is a major priority for universities because obviously if a student drops out, that's, you know, terrible for the individual and it's, it's not great for the school either. And the existing tooling they have to track student success and then move that into a sort of a case management workflow is tricky. It's fragile. I'd say, you know, many schools have some kind of case management tool or CRM they're using and they've in many cases hand arranged or hand built some data transfers from the LMS and maybe from the sis. But those things need to be updated every six months and then it's a call to it once they're installed. It's hard to update them and keep track of, of new evolutions in what's happening in the courses themselves. And the student success agents don't have a great tool to again explore and follow their intuition and say, hey, maybe this thing makes a difference and then track their interventions and see if their intervention was useful or not. Again, not that this doesn't exist, but it's at kind of nascent stages and slightly fragile stages.
And so the ability to put a tool into the hands of the student success agents themselves where they can say, you know, graph me engagement versus zip code and let's see if there's something there.
Or graph me engagement versus first year college students or language they speak and maybe we talk to student services because there's not enough Vietnamese speaking tutors in the writing center. Right.
[00:20:10] Host: I remember I saw Dewey for the first time@educause two years ago and AI was fairly, at least the, you know, it was less than a year into ChatGPT's big reveal and I Saw a Dewey quick little demo and I was like, blown away. It was incredible, like how comprehensive the dashboard was with all the, you know, different data. What is the most surprising feedback you've heard from your Dewey customers so far?
[00:20:36] Guest: Oh, quite a lot. Because, I mean, just first off, the empowering we can do for people, I think, changes their attitude toward data. And again, this goes back to the data culture point, I think, is that if you're just a passive consumer of analytics that are given to you, again, it's not bad, but it changes your relationship to that data when you can start saying, no, I can be an active participant in this conversation with this data. And I think it gives people the sense of actually I can make a difference in my own ability to move this needle that I'm responsible for. And I think that mind shift makes a tremendous difference.
[00:21:21] Host: Yeah, yeah.
[00:21:22] Guest: The second thing I'll say is that the opening up the data has a second interesting consequences that people begin thinking about ways they can use it, that they just, you know, they're in this mode of the only way I can get access is go to it and there's going to be regulations and some of them are valid, don't get me wrong. But it just feels like a tremendous burden somehow. And I was talking to one of our K12 clients and the woman in charge of security was asking, you know, just exploring, hey, how can I take this data now and do better things with it?
And she said, could I give a login to the police station down the road so if there's a lockdown they can log in and see who's in the building? And I said, yeah, that's actually kind of trivial, you know, so, you know, the, the, the idea that now you can actually open this up and think big, big about what you can do with this data rather than thinking small, I think has transformed the thinking of a lot of our clients.
[00:22:30] Host: You mentioned the big data sources in, in the beginning. I think you mentioned a bunch of different ones. It sounds like Dewey is pretty much a platform. You could integrate whatever data sources you wanted to. Right. Whether there's an API or, or LTI or like, what is the limit and what are the most creative either use cases you've seen on types of data or plans that you think could happen that maybe haven't happened yet with like the data sources on a complex environment like a university.
[00:23:01] Guest: Yeah, really, there's no limit. And we're talking to customers now about, I think, something you mentioned earlier about labor force data and sort of outside data Sets that actually don't even belong to the university. But you know, as conversations are happening on strengthening both in higher ed and even in K12, a pipeline to employment. Well, you know, there's labor force databases out there that say, hey, our state needs, you know, 40,000 nurses in the next eight years. How can we start helping people who would be interested and eligible get on those pathways? Absolutely. I mean, we have a demo site up, by the way, that anybody with a edu or.org address can log into just to test out Dewey. We ingested the IPEDS data set, which I hope is still live on a government website.
[00:23:53] Host: Yeah, right. What's going to happen?
[00:23:54] Guest: All that data?
But Anyway, it's at IPEDS Dewey IO, you can go log in. And we just, you know, we wanted a demo site where we didn't have any student data because we didn't want to have to worry about, you know, privacy questions. And so we just downloaded it and you could very easily today. And I've done this for some university clients whose institutional research groups spend, I don't know how many person months a year doing reports of their data versus their peers in the IPEDS data set. I have reproduced Those in about 10 minutes using Dewey, just using the iPad's data set and to show the IR team you know, how easy it is to use and you know how it can be useful for them. So there is no limit. Any data set out there, we can ingest it, we can use AI tooling to help create data marts without a lot of human effort and then create the ability for humans to who are non technical to just start exploring and start reporting.
[00:25:00] Host: Can a school set these connections up themselves? Is it that flexible of a system or do you really need to be involved with setting up those data sources in almost every case?
[00:25:11] Guest: So we do some of the lift and it depends on how far the school is along in already doing some of this work. You know, if they already have a data lake and they have a lot of it already in there, it's very little work. We can just point, Dewey added in. It mostly works, but we also want to do some qa. We run through a pretty rigorous QA process with each client before we turn it on for public use, just to make sure that everything looks as good as it can be before we open it up to our clients.
[00:25:41] Host: So it seems like if you're a school out there that has all this data, you know, it's kind of chaotic. You need to get some insights into it like this is something you really should look at is data do. It's that it's kind of almost hard to really explain on a podcast or visualize what such a powerful piece of software does when you're not so niche where you're doing one thing and I'm a video content management system. But I think that's the price of having such a powerful system is sometimes hard to, at least for me to kind of comprehend, like what exactly are we doing here? Because it's. There's so much possibility. But is that safe to say, like you're a school that has all this, you have this data lake, you're not sure how to. Don't have the time to draw the insights out of it, or you haven't maybe connected all your data sources to it yet. It's probably a good place to start.
[00:26:27] Guest: That's correct. Or maybe you have got insights, but they're limited because the only folks who can really drive that are your limited set of data analysts.
[00:26:37] Host: Yeah, right, Right.
[00:26:39] Guest: My goal in helping to build the Dewey product has always been that data is useful to everyone. It's useful to the people in student services who want to help students get their needs met and make sure they stay in. It's useful to your admissions team who want to make sure they get enough students and students who they know are going to be a good fit. It's useful to your alumni team.
Every single function in the university can do better with more access to data. And if there's one message, I'd say it's that the more you can democratize this and put this in the hands of your people, the better job you'll do.
[00:27:17] Host: Well, I want to finish up the one kind of more fun question here. You're known for being, at least on your LinkedIn and what I've seen, an amateur edtech historian. What trend or shift do you think history will remember from today's ed tech moment?
[00:27:33] Guest: I think, you know, it's interesting, the infrastructure that's been put in place for pretty much the last 20 years, which is the SIS, the LMS and the LMS tools. It has been that way for about 20 years, plus or minus, and certainly under the existing pressures around the role of higher ed in society and what higher ed should do for a country and an economy as those are changing, I think we're going to start to see pressure to have a substantive shift to another kind of architecture going forward, something more. We talked about skills based for a long time, but I think now is the time. It's going to start happening and with, I think, less emphasis on the generic systems that we have currently and into more precise systems that are more facing toward future employment.
[00:28:28] Host: Yeah, I agree with that. I feel like right now we're just testing everything. What can everything do? And there's just so much kind of noise out there. And we'll see in a few years possibly what's what the shakeout is. But thanks for being on the show, Jeff. I'm going to leave notes to Jeff's LinkedIn and to the Dewey website. We'll put those in the show notes. So thanks for being here.
[00:28:48] Guest: That's great. It was a lot of fun.
[00:28:49] Host: Bye.
As we wrap up this episode, remember EdTech Connect is your trusted companion on your journey to enhance education through technology. Whether you're looking to spark student engagement, refine edtech implementation strategies, or stay ahead of the curve in emerging technologies, EdTech Connect brings you the insights you need. Be sure to subscribe on your favorite podcast platform so you never miss an inspiring and informative episode. And while you're there, please leave us a review. Your feedback fuels us to keep bringing you valuable content. For even more resources and connections, head over to edtechconnect.com your hub for edtech reviews, trends and solutions. Until next time, thanks for tuning in.
