[00:00:00] Speaker A: Our formal policy on AI is again to free educators from these tedious administrative tasks so that they can provide that rich, formative feedback to students.
[00:00:22] Speaker B: Welcome to the EdTech Connect podcast, your source for exploring the cutting edge world of educational technology. I'm your host, Jeff Dhillon and I'm excited to bring you insights and inspiration from the brightest minds and innovators shaping the future of education. We'll dive into conversations with leading experts, educators and solution providers who are transforming the learning landscape. Be sure to subscribe and leave a review on your favorite podcast platform so you don't miss an episode. So sit back, relax and let's dive in.
Welcome everybody. I am excited for another podcast, another EdTech Connect podcast called the Future of Grading How Crowdmark Enhances Educator Efficiency and Student Outcomes. And today we have Michelle Cares, who is the co founder and CEO of Crowdmark, an innovative grading and assessment solution for higher education. She has over 20 years of experience in education and technology where she's led teams in six successful startup companies at various stages of growth. Michelle is also the founder of Design ux, a strategy and design firm.
Previously, Michelle was an entrepreneur in residence at the Toronto Metropolitan University pmz, a world leading global accelerator for tech startups. Michelle is a frequent speaker at technology and education conferences and advises a number of startups on strategy, design and customer acquisition. So welcome Michelle and I'd love to hear more about your founder's journey and how you got to Crowdmark.
[00:02:08] Speaker A: Thanks Jeff. I'm really excited to be here today. My journey, it's a question people ask me. I've been asked that more than once.
So my career has always been in either education or technology or the intersection of those.
And I started out in education and then when I got my mba, I did my dissertation on elearning.
So I thought, you know, looking at the Internet and all the different ways that we would benefit from it, I thought education would be a really impactful place.
And it turned out that it was. And then I started my career working mostly in search engine optimization and online marketing. And that was back pre Google IPO. So there were, if you recall, there were about 11 or 12 different search engines out there and we had to optimize our websites, our customers websites for all of them if you remember, like Alta Vista and those kinds of browsers. From there I was the first salesperson hired by Design or Desire to learn or D2L as it's now known. And they've grown exponentially. Probably the biggest edtech success story in Canada.
And then from there I worked in subsequent startups as either. The first three is an employee, the last three is a founder.
And the idea for Crowdmark came from James Colliander. So James was a professor of mathematics at the University of Toronto, and he was tasked with grading the Canadian Open Mathematics Challenge. It's huge math contest.
And he had 70,000 pages of student work to grade over a weekend. And he had a whole team of graders and they had to sort out all of the submissions in stacks of paper and then trade that paper back and forth. He thought that was very time consuming, so he came up with a prototype. The following year, they graded the same contest in half the time using the Crowdmark prototype.
So with the proof of concept established, that's when I joined Crowdmark and helped to bring the business aspect to it. And five months later, we had our first customers.
[00:04:39] Speaker B: Wow, that is. That is really exciting. Well, let's cut. You got right into it. Tell me. Grading is kind of like one of the core functions that faculty have to tackle all the time. Probably not their favorite thing to do. But how is Crowdmark really helping university faculty?
[00:04:54] Speaker A: I think you really nailed it there. Grading is like an essential part of the education system, but most people don't enjoy it.
And so Crowdmark was created to help educators scale their time by automating the tedious administrative parts of grading and assessment in order to free up the educator's time to leave better feedback for students.
It also supports collaboration amongst grading teams. And so if someone creates a really effective comment or creates a rubric that can be shared amongst the whole grading team, and then, you know, student number 500 gets the same thoughtful feedback as student number one. Because, as you know, grading can be more like a endurance exercise.
[00:05:52] Speaker B: If you're graded first, you might get more feedback because the faculty member's not tired yet. Or.
[00:05:57] Speaker A: Yeah, imagine writing the same comment to the same error 500 times. You know, by the end you end up with a check mark. But with Crowdmark, because we have reusable comments and these can be easily applied to the student answers, then you don't have to retype it out or rewrite it out every time. And you can easily place it on the student work so they can understand where they went wrong. So the students are still getting really good feedback, but it's less work for the educator.
[00:06:26] Speaker B: Right, that makes sense. So if it's less work for the educator, can this help academia as a whole? And that maybe classes, big classes, aren't going to be as resource intensive? Maybe.
[00:06:41] Speaker A: Well, what we find is there's, you know, two, two different ways people use Crowdmark. There's the large scale service courses where they could have, we have some courses up to 2,500 students. And you know, in order to tackle that grading process for midterms and final exams, they need an army of teaching assistants to help grade all of those papers. So Crowdmark assists in managing that grading team, providing the rubric for them. And then everyone can start grading in a series, like all the question ones, all the question twos, all the question threes simultaneously. And the software is designed to avoid collisions.
So you're not going to overwrite your colleagues work and you can do it much more efficiently. So we've had some private studies like white papers done comparing traditional grading methods with Crowdmark, and the results are that educators can grade three times faster.
[00:07:51] Speaker B: Wow, that's incredible.
[00:07:53] Speaker A: So with that time savings that could be spent on research or maybe spending more time with students.
[00:08:00] Speaker B: Well, so I have to ask, and we might as well just talk about this early, how are you leveraging AI, if at all?
[00:08:08] Speaker A: Yeah, AI is the flavor of the day. Interestingly, Crowdmark has been using artificial intelligence since 2016.
And our formal policy on AI is again to free educators from these tedious administrative tasks so that they can provide that rich formative feedback to students. So we do not want to replace educators by grading with AI and so we will never build something to replace that human to human connection.
We see teaching and learn as fundamentally human.
However, we're going to leverage AI to automate different things. So one of the things we started with was optical character recognition. And we can read the student handwriting on the COVID page and then automatically match their exam to the student identity in the roster. So now that's a big part of the grading process you don't have to worry about. Once you have those papers identified, then of course you can export the grades very easily back to the lms. You can do statistical analysis on student performance.
Other things we've done is including optical marks recognition. So this has been in education for a long time where you can use a multiple choice bubble sheet and that gets graded automatically. These are a couple of examples of how we're using AI to augment grading but not replace it.
[00:09:42] Speaker B: Right. That makes sense. Seems like a perfect use case for AI actually.
What are some of the challenges, the early challenges you face in convincing educational institutions with such a critical task as grading, to try something new.
[00:09:59] Speaker A: Try something new. So our biggest challenge and our strongest competitor even today is the status quo.
So it's not only that change is hard, but educators are inundated with ed tech solutions with big promises and disappointing results.
And this has happened over the last 20 years. Everyone, everyone is talking was and continues to talk about personalized learning. And yet that still hasn't become a reality. And so I think people are a little jaded. So they want to keep doing things the way that they've done them and they're a little tired. They not necessarily want to learn something new.
But the reason we've been successful is because we offer a practical solution to a persistent challenge, which is grading. And we aren't asking anyone to change the way that they assess students because they can reuse their exam templates, they can set up their exams and follow the same practices as they always did. Imagine a large room with 250 students sitting down and writing their answers on paper.
Where Crowdmark comes in is in the back end of that, where you scan. Excuse me, you scan all that student work, upload it to Crowdmark, and then grade it online. And then as soon as people try it, they say, oh my goodness, I can't imagine going back.
Yeah, so it's getting over the hurdle of just try it.
[00:11:42] Speaker B: Yeah, yeah, yeah. Seems like so much out there these days that it's almost hard to find the right solutions. But we don't, we all don't need another place to log in like, oh, is this another system? Does it matter what LMS is being used? Do you have connections to the big, you know, Canvas or Bright, you know, Brightspace or Blackboard?
[00:12:03] Speaker A: Yes. Crowdmark is partnered with all of the five major learning management systems and we offer single sign on. And one of the things to keep in mind, though, is we're not a plugin within the lms.
[00:12:17] Speaker B: Okay.
[00:12:18] Speaker A: So we have, you can sign in and you can import student rosters and export grades. But all of the magic happens within the Crowdmark application itself.
[00:12:30] Speaker B: When I think of grading, I think there's two things I think of. One is there's a nuance of subjective grading. There's essays or there's feedback you need to provide.
Can you talk about the nuances of subjective grading, like essays or creative assignments versus the actual grading part of it?
[00:12:50] Speaker A: Yeah. Okay, so I'm going to take that one step further and talk about STEM versus non STEM courses.
So Crowdmark is, was originally and is very suited to grading STEM courses, particularly mathematics, physics, economics, chemistry. And the reason for this is the student's response often has to be handwritten, and that's what's really special. About Crowdmark versus any other assessment software out there is that it enables digital grading of handwritten work, even if the student, you know, if the student writes it in person or if the student's doing an online assessment, they can write their answer on a piece of paper and take a photo with a mobile phone and upload the image and that can get graded online.
So Cred Marks excels in that area. When it comes to the actual grading part, it's either right or wrong and you can get a plus or a minus mark. There are nuances to like the steps you take where you can add more feedback with the student for subjective areas like essays.
You know, they're looking for a rubric type of approach. And that way you want to organize your comments and make sure that you touch on all the different ways that you're evaluating that student throughout their response. And of course, in 2024, all those responses are typed and uploaded as a PDF, so they already have an electronic version, whether they want to grade it on Crowdmark or grade it somewhere else. Crowdmark still offers all the collaborative grading tools and the reusable comment library. So there is time to be saved there. But regardless of what the subject area is, it, it's really up to the instructor on how they want to grade it.
[00:14:48] Speaker B: Right. Right. It kind of aligns with. I just came back from an a, from an AI conference and there's this big conversation about, like, how do we govern AI? And there's no governance out there and all the governance is often just pushed down to the faculty member. Right. Even at K12, higher, higher ed, like they're saying, well, just a faculty member can decide that. So I, I see that as an opportunity. Like people get scared. But I think it's at that level now for faculty to really adopt this.
What I'm really curious what your thoughts are on the ethical implications of AI in education, particularly about the data privacy and the academic integrity.
[00:15:27] Speaker A: Good question. First of all, we have to be really careful about how AI is implemented and for what purpose. Purpose in the pandemic. So I've talked a lot about paper based assessments. So during the pandemic, Crowdmark pivoted and we extended our online exam workflow and pushed out like 15 features in six months. It was rapid. You know, everyone was just trying to find a solution to help them get through that the end of the year and into the next year.
And because people didn't have time to really think through their assessment process, there was a lot of concern about academic integrity.
And some institutions implemented online proctoring solutions. And a lot of these online proctoring solutions were driven by or powered by AI.
And then we discovered unfortunately that there was some inherent bias built into these tools that flagged people who actually hadn't done anything wrong, or people from disadvantaged backgrounds, or people from. Who just couldn't handle the pressure cooker of knowing that something's watching them while they're trying to complete their exam. And in many cases it just had a really bad outcome for students. So we really have to be careful not to implement surveillance and not to use AI in a punitive way. I think AI should be used in a supporting role and I think we have to also be very careful about protecting intellectual property. And that's something that we've thought about quite a bit with Crowdmark. Because once your course is built in, in Crowdmark, your assessment, the comments you and your teams use to create and share amongst your grading teams and all of the work that the student submits remains the property of the student and remains the property of the instructor and the university.
So we don't claim ownership of any of that. Whereas you'll find other software out there where students are asked to submit their work, which then is reused to check other students work. But no one is giving credit back to the student who made the original work.
[00:18:01] Speaker B: Right. You just answered one of my other questions about who owns the work, so.
[00:18:05] Speaker A: Thanks for that, but you're welcome.
[00:18:08] Speaker B: It comes up a lot, a little bit of a tactical question like how does it, what does it look like if a school wants to try Crowdmark? Is what's the onboarding look like? Is it just one faculty who might try? This is. Do you have models where it's. It's kind of an enterprise to the university or directly to faculty? What is that?
[00:18:28] Speaker A: We're very flexible with that. Crowdmark tends to start with one or a group of instructors, for example, say in a math department and they'll try out Crowdmark for one term and then share their feedback with their colleagues. Then we would expand to multiple terms or perhaps to a faculty or a college within an institution.
And then as it continues to grow across campus, then we look at an enterprise license. But we're here to support, you know, individuals, instructors all the way up to university. Wide adoption of Crowdmark.
[00:19:05] Speaker B: It's. I've seen it be. It's a really clever and successful strategy to really go right to the faculty. And all of a sudden maybe some leaders are like, hey, we have 20 faculty using this product. Maybe we should get a site license. It's the proof is in the pudding, all these people using it.
I have a really good question I want to ask. Here is if you've discovered any unexpected outcomes or success stories from institutions that are using Crowdmark.
[00:19:33] Speaker A: Yeah, I have some cool stories, actually. When we first started Crowdmark, I would say we were about a year and a half in to having customers, and one of our instructors sent us a photo of himself in the Arctic. He actually angled the camera so he could see the screen of his computer, his laptop, and he was grading student work while he was doing research in the Arctic.
So that. That really was rewarding for us because the idea was to enable instructors to grade anywhere, anytime, and he totally validated that.
And then there was another story that came from some instructors around. It was around the Christmas holidays, and they were on a train heading back to their hometowns, and they were sitting with their iPads and they were grading papers, and they were sipping a glass of wine, and they were quite pleased that they now could do this. And then the train stopped, and another instructor from a different institution came on with two stacks of papers underneath their arms and juggling their suitcase and trying to find their seat. And the instructors even took a moment to write us an email and say thank you for Crowdmark, because that very easily would have been us before we started using Crowdmark. And then the final story I have for you, I just heard this one this year.
So within Crowdmark, instructors can write comments for students and the students receive them, but students can also write on their exam papers a message to their instructor if they wanted.
And one student wrote a message to their instructor on their final exam. And when the TA was marking it, he tagged it, so you can tag an answer and crowd mark for the instructor to review. So he tagged the answer, the instructor went in to review it, and the student had written, thank you for believing in me. You. You're the only one that has. And I really appreciate all your help this year.
[00:21:45] Speaker B: That's got to be really rewarding. I love hearing those stories.
[00:21:49] Speaker A: Yeah, they're like goosebumps. When I heard it from the instructor, I was like, oh, that's. You know, that's the essence of what we've been trying to do in our mission is to create a dialogue between instructors and students, and it's happening.
[00:22:04] Speaker B: Yeah, it makes you feel good about being in education, too, you know?
So kind of a final question for you, and it's really just, what advice would you give to other entrepreneurs looking to innovate in the. Maybe in the intersection of the future of technology and education.
[00:22:23] Speaker A: You know, that's. That's a good question as well. So I've heard, I heard from OpenAI, it's very famous. And the chief operating officer of OpenAI talked about how AI would transform education and everyone would have a personalized tutor and it would personalize education at scale and solve all of our problems. And we've heard that over and over and over again from edtech entrepreneurs, from the MOOCs, if you remember that time period, to personalized learning with tutoring and on and on.
My advice to entrepreneurs would be to work closely with educators to define a very specific problem that needs to be solved, solve that problem and go from there. Don't try to create a solution to solve all the problems. Solve a small one and build from there and actually build something people want to use.
[00:23:39] Speaker B: That's excellent advice. Well, I want to thank you for being on the show, Michelle. You can find more out about
[email protected] or is there any other place you want someone to connect with you, Michelle?
[00:23:51] Speaker A: I totally recommend going to our website. We also have a blog on our website with some views on education and AI and where it's all going.
[00:24:03] Speaker B: And you can also find crowdmark listed on edtechconnect.com so thank you for your time, Michelle, and I'm looking forward to following Crowd mark down the road here.
[00:24:12] Speaker A: Thank you. It's my pleasure.
Sa.
