Episode Transcript
[00:00:00] Guest: Institutions. Very often you have a few professors in there that have a bit more time and they come out with great ideas. So institutions will come in and it always comes down to what are my peers doing, what are my peers using, what are my peers going after or trying to look at?
[00:00:24] Host: Welcome to the EdTechConnect podcast, your source for exploring the cutting edge world of educational technology. I'm your host, Jeff Dhillon, and I'm excited to bring you insights and inspiration from the brightest minds and innovators shaping the future of education. We'll dive into conversations with leading experts, educators, and solution providers who are transforming the learning landscape. Be sure to subscribe and leave a review on your favorite podcast platform so you don't miss an episode. So sit back, relax, and let's dive in.
Today we have Justin Menard. Justin is the CEO of ListEdTech, a company that provides comprehensive data on education technology systems used in higher ed. With over 20 years of experience in the field, Justin is passionate about using data to simplify decision making for his clients. Justin's experience in education technology stems from his extensive background in higher education, including 23 years at the University of Ottawa. His roles there range from systems Manager to Senior Business Intelligence Analyst, providing him with a deep understanding of the needs and challenges universities face in adopting technology. Driven by his desire to fill a gap in the market, Justin founded List EdTech in 2014. The company's mission is to empower decision making by providing the largest collection of data of higher education systems systems. Justin's insights are sought after, as evidenced by his quotes in publications like the New York Times, Inside Higher Ed, and the Chronicle of Higher Education. He's also a contributor to the List EdTech blog, where he offers in depth analyses of market trends and case studies. So welcome to the show, Justin.
[00:02:17] Guest: Thanks for having me, Jeff. It's great to be here.
[00:02:19] Host: So I've met you a couple years ago. It was really brief. Since then you've really I'm really impressed with this happened and I don't think I've told you this, but in my monthly newsletter at TechConnect monthly newsletter I often link to your blog stories, your news articles that you're writing. So I'm really a big fan of your data and someone even told me once, have you seen ListEdTech's beautiful data? They call it beautiful data. So I'm like, yes, I'm aware of their data. It's incredible.
[00:02:46] Guest: So well that's really appreciated. I always feel like we do everything we can and you're always trying to build on Whatever you have. So, yeah, it was great meeting you a few years ago. And I've sort of been looking at your site on the side every once in a while, going, okay, yeah, they're growing.
[00:03:05] Host: We've evolved. You've been pretty focused and like. And we're. We've done a lot and so. Yeah. But I want to start with your Bio. With your 20 years of experience in educational technology, what sparked your interest in this field?
[00:03:20] Guest: I've always been very curious. I love data. I don't know why I always go back to data, but there's nothing really that triggered everything. It's just experiences that you have that you sort of accumulate and you go where nature brings you type of thing. So that's me.
Studied far too long at university, not knowing what to do with my life, and it sort of brought me into technology. So. Yeah.
[00:03:45] Host: So is it fair to say you've kind of always been a data guy and always had this kind of craving or a knack for data?
[00:03:51] Guest: It's sort of. And this might get personal fast, but I'm dyslexic, so automatically everything that has to do with data is more comfortable for me. So that's probably why.
[00:04:01] Host: Yeah, yeah, makes sense. Yeah. And your background isn't solely focused on education. You've done a lot of travel. Tell us about the impact of that, your global reach with the potential of edtech.
[00:04:13] Guest: You're right. Travel has always been very important for my family. Being Canadian, we get longer vacations than American.
Phil Hill, who you. You probably know, is always telling me that we have too much vacation because we're socialist.
He's always making a joke about it. We usually take three to four weeks vacation time to travel. And Phil is always going. We took a Canadian vacation. We took eight days.
So definitely it's important for us. And we've been lucky enough to actually take a year off with our kids and travel all, all over the world. So that was something that I'm happy about. It was a great way to get to reconnect with my kids. And while we were doing this, I was forming the company in my head. It had already started. And I remember traveling. We were, I think in Berlin at that time. And I was writing my blog as I kept on going to try to figure out, is there interest in this company? And a guy was answering me and going, yeah, yeah, we use Moodle also, blah, blah, blah, type of thing. And I was like, there's something in what he said that I went back and said, where are you? And he Said, I'm in Berlin.
So am I. So we went to have a beer and that was my first introduction to sort of like most institutions have the same tech stack. That was the first piece of the puzzle there.
[00:05:36] Host: Yeah, yeah. So I'm looking at your site now. This is just for the audience here. The articles that you write that you. They're even public, they're not behind any firewall or anything. And one of them you've been doing for the last few years, I'm not sure how long you've been doing it was actually. It's probably in there. Might have been doing this for a decade. Educause 2024. Tracking the evolution of higher ed tech through exhibitor data. So what he's doing here, Everyone is. Everyone has a big tech conference, you know, in the country that travels all over the. It's in a different place every year, but it's in October, it's called Educause. And some of the booths are these 10 by 10 booths, but some are these monstrous booths. And so we can see data and it represents maybe the evolution of the market and, you know, what are the booth sides of different industries? So that's one of the articles on there. There's one called what's the Market Saturation look like in Higher Ed? All sorts of. All sorts of different articles. But I'm always trying to check this at least a few times a week. Tell me a little bit about how you. I mean that one specifically about the Educause data. But what inspires you to do different kind of creative data? Is that coming from your head or like.
[00:06:43] Guest: No, nothing really comes from my head. It's usually conversations that I have with very smart people. As you know, being in this business, you always get encouraged to work or push in a certain direction. And either institutions or vendors always have great questions. The article that you mentioned about Educause, I think that was it started out with a conversation with Charlie Moran, great guy Moran technology. And I was at his booth and he. I don't know, he hinted at something. He mentioned something that was sort of in the direction of, hey, have you noticed this company, their booth size is, I don't know, this size this year, but last year it had the massive size. So what do you think that means? They think they might be having issues or what's going on. And I remember thinking, that's interesting. And I came back and I started to play with the data because they had like three, four years of historical data. We compiled it and that is a fun post to do because it brings up so many conversations and you go around and people are going, oh, that post. Did you notice them, them this year?
[00:07:52] Host: And I'm like, yeah, what happened to X Men? They didn't even show up or.
[00:07:56] Guest: And very often when these vendors get to this really big boot size, it's hard for them to go back once you go to the big size, because everybody's looking at you going, oh yeah, not looking good. So, yeah, it's a big risk to take. And these things are not cheap.
[00:08:12] Host: Yeah, yeah, yeah. I even have some confirmation. Some of my clients, they go to educause, they ask me, should we go, should we go? And it's. If you really, if that's your market, it's almost sometimes the price of doing business and they want deals out of it. And like, I get that, but I had to see a couple of CIOs tell me they said just what you're saying, Justin. I say we come year after year just to see who keeps showing up and who doesn't, you know, so.
[00:08:35] Guest: And the vendor hall, I don't know if you were there this year, but the vendor hall this year was incredible. The atmosphere, people, it was as much as last year was really interesting and fascinating. This year was like back to the, the 1918 type of thing. At the peak.
[00:08:53] Host: Yeah, yeah, there was over 8,000 people, I think they said the second largest ever. They'd split it up in this quadrant or like three different areas of different technology and it was, it was quite massive.
[00:09:04] Guest: Yeah, I like the neighborhoods. Yeah, that was kind of interesting because you could sort of go, okay, I'm here. Oh, the other vendors that are sort of like them in the same area. I like that. Yeah.
[00:09:14] Host: So you've clearly dedicated a large portion of your career to higher ed, spending 23 years at university of Ottawa. Could you share some highlights from your time there that maybe helped shape your perspective on technology and education?
[00:09:27] Guest: I've been really lucky in my career because I, as I mentioned, I studied. I think it was six years to do an undergrad degree full time, switching majors almost every session, not knowing what I was going to do. So I, I'm just a very curious person. I'll try anything. And after six years of studying, I finally got a degree in political science. So nothing with technology. And then I went to do my teacher's college and finished teachers college and the university offered me a job and I stayed. And it was always in the technology trends. And I've been lucky enough to have these great bosses that would always Give me enough rope to try stuff and not kill myself.
[00:10:12] Host: Yeah, that's great.
[00:10:13] Guest: I was always able to form these great teams of really weird people that would come together and we would always do special projects. And I've had so much fun. So Universal Ottawa was really good for that. There's so many projects. I was only doing projects, so it was a really great time there.
[00:10:31] Host: So tell me what exactly is ListEdTech and I can kind of.
[00:10:36] Guest: You.
[00:10:36] Host: I'll give you my perspective what I think it is and you might be interested in that. You know, you have this data portal, then you have these ad hoc reports that people can purchase separately. But like, are we targeting higher ed or the vendors or both? Or give me an overview of what, what you're doing.
[00:10:51] Guest: So when we first started the company, my goal was to help universities and colleges. That was give them actual data that they can actually figure out who's using what product. That was the main question. I think two months in, we had a first vendor come in and we sort of saw, oh, the vendors want this information even more. And then you're going, okay, let's pivot where the money's going up. So started working with vendors. And we did that for maybe six, seven years. And about two years ago, we sort of came back to institutions because that was a different type of same data, but being used differently. So we help vendors sort of better identify potential clients, better understand their fields, better understand how, how their product is being used with what products also. So you can have a very specific product, but the network sort of shows you different products that are always used at the same time. So that's interesting for them, for the vendors, for the institution's point of view. It's a lot about what are my peers doing? We're looking for a specific product system, whatever it is, and we're using products. Abc, who else is using these products? And we can show them what other vendor they should be looking at. So we're never recommending a product or doing anything like that. It's more about what our peers are doing. Because as you know, and you're in the same type of business with EdTechConnect. One of the first things I remember having experience that we were looking for a CRM and you Google it and you sort of land on G2 type of thing. And you go in and they give you a list and you write your email and then they give you a list of CRMs. And you're looking at these and you're going, the more you dig, the more you sort of notice none of these are actually in higher ed.
So that was sort of a clue there that there was something more. So why not show data of products that are only being used in higher ed? Yeah, yeah, there was that. And just to go back to educause, the trigger for the company was we were looking for a new student information system. That was the root of all this. We had a homegrown system and the code that we were using was something that was hard to support. And I remember going to the. To educause. We were members and downloading the info. They do a great survey every year. They still do it now. I think it's getting smaller, though. And I remember downloading everything and there was like, I don't know, let's say 500 institutions. And I was looking, it was written University of Ottawa. And it was written Banner school. And we're like, not a banner school. Why is it banner? I go see your CIO and he's like, look. I give it to my assistant. She fills out the boxes that she thinks it is. So that was my first clue of maybe there's a better way. And even though I love educause, I've been to, I don't know, 17 conferences, I wanted to figure out a better way. So from a survey that gives you maybe 500 institutions, who's using what? We have multiples of 10 and everything else. So for LMS, I think it's our biggest category. We have 17,000 higher ed institutions that we know what they're using. I think it's around 20,000 school districts in North America. So you can really dig into the data and figure out who's using what.
[00:14:14] Host: And so, okay, 17,000 universities, you're talking globally now.
[00:14:18] Guest: Yes.
[00:14:19] Host: And when you say districts, are you. So you're doing K12 in the US.
[00:14:21] Guest: Or is that you're doing K12, US and Canada only? Yeah, those are the only two.
[00:14:28] Host: So you're bringing up too, like, we have similar missions. Like, When I started EdTech, it was really. The idea is crowdsourced. I think the difference is how we're doing it. Do you have just a bunch of analysts everywhere? Is that how you're doing this?
[00:14:40] Guest: I remember the first two years it was media wiki that we had, because every time we find a piece of information, we need to write where the source is so we can validate it every year to make sure they're still using the product. The wiki at the time was wide open and anybody could add information or remove it or correct it. That was the initial goal or the way it was processing. When we hit year three type of thing, we had so many people scraping our site that we were like, so, yeah, firewall came in and we restructured. So how we find our data is we only have data that's public, so it needs to be a public source. And we have a bunch of different scrapers that go out every day. These scrapers do everything from university sites, company sites, contracts, RFPs, boards. We love boards, boards, professors complaining. I love it. A student who's on Twitter complaining about a specific product that their university is using. These are all clues that we grab. So we get about 25,000, we call them alerts per day. We get machine learning that removes most of it. And we have 16 people actually manually go through the info and confirm what they're using. At first, we wanted to do everything as any company wants to. Let's automate everything. It's the best way. And we soon realized there needed to be a manual part of it to validate the information because there's so many variables. It's not because a student says, I hate Product abc, that they're actually using Product abc. They could be using it somewhere else or, I don't know, they have a side project that they're running.
[00:16:22] Host: Yeah, that's. That's great to know, like, how you're doing it, because it's. It does seem like a challenge that. I don't know how you keep up, but you. You gotta have some automation in there to make it scale that well.
[00:16:31] Guest: There's a lot of automation, but there's a lot of manual work that goes into it. And we have just the. The data quality is so heavy to. To manage. Yeah, I remember at first we're struggling trying to figure out how sure everything's up to date constantly. And it was a long process for us to go from doing spot checks every day to now we do. I think it's 3,000 variables that we test every month. And we were trying to always be over 95% accurate every year. You have to go back, validating that they're still using the info. It's not because a contract says that they're using Product abc, that the university is actually using it. It could have been just thrown in with everything else that they. There's a lot of manual work that goes into it.
[00:17:14] Host: The data I imagine you're creating, I haven't seen anything that's behind your portal or anything, just your public stuff. It's got to be pretty valuable to so Many people. What are some use cases that people you've seen universities or even vendors use your, your data to help them?
[00:17:31] Guest: I think the most stressful, a few years ago, the first time that Instructure went for sale, there was I think three companies sort of bidding. I think there were four bidding to try to figure out if they were going to buy this. And we were part of the mix. Three of the four companies were buying our data, the raw data and going through it. And I remember being so stressed out going, either this works or this is when we know if we're going forward. And it worked out of course, because we're still here. But that was the first time that we sort of figured out okay, then there's the. It's great, we have the companies, institutions, we knew that was something down the line. But then you had all these big firms that were coming in to get the data for very specific reasons. So that, that was another part of the puzzle that we, I hadn't thought about but yeah, worked out really well.
[00:18:21] Host: So it's got to be really kind of exciting to have that much attention but also a little bit nerve wracking to know that you're using this to make such big decisions.
[00:18:31] Guest: We got better lawyers back then that, that was the first thing we sort of went through. We going, okay, make sure this is in order.
[00:18:38] Host: Yeah, well, my problem, like one of the challenges that I think I've been building up for this question waiting like not, not hitting the, in the beginning with it, but the, the evolution of categories themselves in higher ed. So we used to have the CMS and the SAS and the LMS and the ERP and I named them all there pretty much. I mean there was a, there was a handful, but now even, not even how we label them, you know, we should write something on the evolution of the acronym in higher ed. But the categories themselves, like student engagement platform five years ago is different than the student engagement platform now or personalization tools or these things that are, you know, companies are doing so many things. How do you categorize the evolution and keep up with the sheer. You know, I never even knew these types of things existed with AI and I don't even know how to ask the question, but how do you keep up with the evolution right now?
[00:19:33] Guest: I wish I had a clear answer. And you're right. I remember the two product groups that we led with were SIS and LMS and like if we can do these two, we're good to go. And I think we have 30 products that are live in the portal and we have over 80 that we're working on in the back. And very often, like, we're great at finding information, but as you sort of said or hinted to, is this an sis? Is this a continuing education system? Is this. And you're going, well, I'm not an expert in this. So we go and we actually get vendors to help us out. Look at the information. Did we tag it properly? Are we missing anything? So that's sort of the last piece of the puzzle. We ask vendors, there's consultants that help us. There's a bunch of different people that are much smarter than we are going out and helping Validator information. And having so many vendors sign up, they're always looking at their data and they're saying, oh, this guy shouldn't be in here because we'd be number one. If you remove this one, then you go, yeah, maybe there might be another reason why you don't want this guy here. But these great discussions that we have with the vendors that are always help us out and as you said, product categories, they multiply. Or vendors are saying, well, we're unique. There should be just products. This, this new product category. And it's just us. And you're going, yeah, well, that's not going to work out.
[00:20:59] Host: And it's great to think. I just looked at a demo of a tool out there that does this collaboration tool for, like, images, collaborate on text. It also does grading, it does workflow. I'm like, okay, it's great you do all this, but. So I could almost talk myself through it and say to you, Justin, is like, when you're doing a grading analysis of grading tools out there, this one maybe would make the list for that, but it would just make the list based on the problem you're investigating or that space. And that's maybe how we need to tackle it, is things are emerging now where collaboration means a million different things now.
[00:21:30] Guest: Yeah. But we force each product to be in one product. One product. Or they can't be multiple because you have these great products like Salesforce, They're a CRM, they could be a retention tool, it could be an sis, it could be anything you want because there's so many plugins. So we sort of force them in. What's your main goal here? What are you doing?
[00:21:51] Host: So, yeah, we do that too. Yeah, you have to pick one category to live in and tag it for your use cases. Yeah. How about on the university side? What are universities using your data for? I mean, I can imagine some things, but I'd like to hear what you think.
[00:22:06] Guest: Yeah, so it's funny because I sort of mentioned about a year and a half ago, we opened it up to institutions and we sort of had the same basic info that we were giving to vendors. But institutions, very often you have a few professors in there that have a bit more time and they come out with great ideas. So institutions will come in and it always comes down to what are my peers doing, what are my peers using, what are my peers going after or trying to look at? So a lot of our information is sort of, you go in and you see what your peers are doing, you go in and you can actually look at your peer, you can create your peer group, you can look up a specific institution, see their whole tech stack and you go, oh, they're using this, I didn't know that. Or they're moving towards this. Or you can actually go in and look at RFPs. So we have thousands of RFPs. They go into and say, oh, I didn't know cloud was so important for this. Maybe we should make sure that we include that in there. So institutions, their network of products is something that's very important and we sort of help them identify which products maybe are outliers for their specific tech stack. So it's not because these are not great products, it's just for your certain tech stack, the fit might not be perfectly there. So we do a lot of network work.
[00:23:29] Host: Have you had requests or do you offer any sort of consulting services or is it a complete, complete self service type of product?
[00:23:36] Guest: So we do a little, we're sort of moving away from that and we're getting vendors, great companies that specialize in this. So one of the things we offer is sort of subscribe with us, get the premium. We can answer most of your question if there's something specific that you want. Let's say you mentioned Phil Hill, you want questions on lms. Well, we have an arrangement with him. If you want specific questions, why not go to the man who understands it more than anybody else or the opm, Same thing. Or if you're looking for ERP or stuff like that, we have an arrangement with Charlie Moran. So that we have. People are very neutral, very well known and much smarter than I am. So they can answer the question.
[00:24:23] Host: Well, I'd love to hear what you're thinking of for AI. How is, how are people coming to you with.
They often don't have a great question as like, which AI tool should I use? How is AI going to change our industry? How Is that affecting you right now, Justin?
[00:24:39] Guest: The way we sort of answer it is more looking at the vendor roadmaps because there's not, as you sort of mentioned, there's not a product that's going to change everything else. The chat bots very often are in the mix, but it's more of a plugin type of thing that institutions or products are going after. So we like to see the different strategies that the vendors are bringing along. Some are betting everything on it because they have an ecosystem that can sort of feed data from multiple sources. Others sort of partner with great vendors that they specialize. The AWS and the Microsofts and whoever else. They sort of have that infrastructure there. So they're partnering up. I remember when it first came out, all the vendors are like, well, we're going to build our own LLMs. They soon realized that this is not a model that we can compete in, so they're partnering up. That's to me is more interesting to sort of see. If you partner with this company, maybe you're looking at it from a certain angle. If you're partnering with another one, it's something else that you get out of it.
[00:25:41] Host: Yeah. Well, I think I have one more question for you and I would say like, what are some common pitfalls you see institutions make when it comes to tech adoption and how are you helping assist in those scenarios?
[00:25:54] Guest: I remember when I was at University of Ottawa, I did a lot of mistakes selecting products because you hear stuff that's really great. I think the best thing that we can do for institutions is sort of point them at peers that have been through it before. And that's one of the features that we have in our portal is let's say you're looking for a specific product. We give you a list of 10 institutions that use the same tech stack as you and say they're also using this product and here are three that moved away from this product. I'd rather point them to resources of their peers that are either using it or moved away and they can reach out and say, why did you move away from this? That's to me is a better resource than me just explaining what I think is happening.
[00:26:40] Host: I use some of that strategy as well. I think people higher ed loves to be on the the leading edge. Not quite the bleeding edge though. Well, that was really fun, Justin. If you want to get a hold of Justin or List EdTech, we will put those links in the show notes and I just want to thank you for being on the show.
[00:26:58] Guest: Thank you, Jeff.
[00:26:59] Host: And Have a great rest of your day as we wrap up this episode. Remember, EdTech Connect is your trusted companion on your journey to enhance education through technology. Whether you're looking to spark student engagement, refine edtech implementation strategies, or stay ahead of the curve in emerging technologies, EdTech Connect brings you the insights you need. Be sure to subscribe on your favorite podcast platform so you never miss an inspiring and informative episode. And while you're there, please leave us a review. Your feedback fuels us to keep bringing you valuable content. For even more resources and connections, head over to edtechconnect.com your hub for edtech reviews, trends and solutions. Until next time, thanks for tuning in.
