Episode Transcript
[00:00:00] guest: Because right now we have enough AI tools to really clone our personality completely. We can create someone's digital clone if they give us enough data. Is higher ed willing to cross the line and create digital clones of the students? Like I said, don't, because you can. Probably they don't. Students probably at that age don't care. Most of the students in that age group, they don't care what happens to their data. And that's concerning.
[00:00:26] host: Welcome to the EdTechConnect podcast, your source for exploring the cutting edge world of educational technology technology. I'm your host, Jeff Dillon, and I'm excited to bring you insights and inspiration from the brightest minds and innovators shaping the future of education. We'll dive into conversations with leading experts, educators and solution providers who are transforming the learning landscape. Be sure to subscribe and leave a review on your favorite podcast platform so.
[00:00:54] host: You don't miss an episode.
[00:00:56] host: So sit back, relax, and let's dive in.
[00:01:02] host: Shogoto Chatterjee is an AI and machine learning expert who's built a career helping businesses and institutions make sense of complex technology. He specializes in turning cutting edge AI research into real world solutions that work without the unnecessary hype. His journey into AI started in academia with a PhD in Physics from Arizona State University. He spent years researching black holes before shifting his focus to machine learning. His academic background gave him a deep appreciation for data driven insights, a skill he later applied to building AI solutions for major organizations like Apple, Cisco and Walgreens. While working as a graduate research associate, he developed data models that analyzed complex astronomical phenomena and that translated seamlessly into AI applications, including predictive analytics and machine learning pipelines. His work in AI now focuses on making these technologies more accessible, efficient and effective for a wide range of industries. Today he leads AI and machine learning initiatives at Tensor Product, developing innovative tools that push the boundaries of AI applications. He's also the host of Machine Learning.
[00:02:18] host: Made Simple, a podcast that breaks down.
[00:02:20] host: AI trends, research and deployment strategies for professionals who want practical insights without the fluff. His expertise in retrieval, augmented generation, large language models, and AI strategy makes him a perfect guest to explore how higher ed can navigate the fast changing world of AI.
All right, so today we have Sugar.
[00:02:44] host: To sugar to welcome to the show.
[00:02:46] host: Great to have you.
[00:02:48] guest: Thank you for having me, Jeff.
[00:02:50] host: So I've known you for many years now and I didn't say this in your intro, but you really helped me.
[00:02:57] host: In the beginning of EdTech Connect with.
[00:03:00] host: You know, I was trying to pick my tech stack and we landed on Python. You even helped Me with some of that. So I really was kind of learning your skill set then. And I think you were teaching at ASU at the time. I wanted to thank you for that and then really jump into your career. You started in theoretical physics, then shifted into AI machine learning. What was the turning point that led you to this field?
[00:03:26] guest: So at that point I was teaching at a few different places and also kind of like migrating towards Python development.
So I was teaching for this, that was part time for this small college. And the student retention was a major challenge at that college I was working for. So one day the CEO, she attended a recent workshop and AI was like gaining traction and she came to me and said, hey, I just attended a workshop and where they sincerely said that the machine learning can solve this retention problem that we have. And I want you that you are the only guy with statistics knowledge to look into it. And also programming knowledge, you're the IT guy. So. So that conversation essentially changed my course of my career. So that one single conversation. So I first I dove into the research and then I started learning and then I never looked back from there. Over time, I kind of became increasingly drawn to the cutting edge research and that's what I do my podcast and like mostly cutting edge, it's very niche. But then I also got interested in the operational enterprise level aspects. And then once I learned all that, then I started doing podcasts and like making it simpler for people and more accessible for people. And I also like to read a lot of research papers. I come from a research background, so I'm in between, I'm the go between. I'm the middleware, like in enterprise language, I'm the middleware between research papers and people who want to implement AI. And my podcast is free, anyone can use it and that's what I do right now. So it's called Machine Learning Made Simple. I think you already mentioned that during your introduction. It's growing and we have quite a few viewers and I get feedback and they was like, well, let's discuss Manus AI. Why don't you discuss this new diffusion large language model that came out that can generate code almost instantly. So I take that feedback and then the podcast is very, very cutting edge. We do talk about new topics every week because the research is growing exponentially in machine learning right now. I was talking to someone yesterday and that person does work in quantum computing and does research in quantum computing. Even one year ago. If you look at people are talking about the agentic framework now and it's like they want to implement it. The first agentic framework paper came out in July of 2023. It was not even implemented. The first time it was implemented was July of 2024. A year later, and now we are not even eight or nine months later, people are actually going towards fully autonomous agentic frameworks. And there's a research paper that I just discussed two weeks back in my podcast, where 6G network will be fully autonomous using agentic AI that takes control and replaces human operators. So any kind of telecommunications network, the middle layer would be all AI agents working autonomously.
[00:06:18] host: I've been following your LinkedIn and that's what I've been interested in is your updates on how you're keeping up with the speed of what's happened in the last really in the last few months, in a year, how you mentioned agentic. It's really been just in the last six months or so. I feel like it's really come to the public. AI is everywhere now and we're going to talk a lot about that today. But there's a lot of noise. I feel like. What do you think is the biggest misconception people have about AI today?
[00:06:46] guest: There are a couple of things essentially when it comes to AI, there are two kinds of people. People who are way too optimistic and think AGI is already here. And then there are people who are completely oblivious. They don't care about AI. It doesn't matter to us. So one of the misconceptions I find is that people think that AI can solve everything, like it can solve a few things. But a couple of the things is like hype. And it comes from the scaling effects people have seen. In one year, ChatGPT has grown from 350 billion parameter model to 1.6 trillion parameter models. So that's a massive model and comes with some emergent properties. And that's what really confused people. And people thought, oh, I'm seeing human level reasoning capabilities. Are we seeing sentience? Are we there yet? But that's no. I mean, if you look at Yann Lecun and the way he has presented the topic, these are all auto regressive models. Yes. Transformers has long range correlation, so it can understand semantic representation very well. I mean, it's somehow it mimics our functionality of our brains.
But the issue is that people think it will solve everything, like it will have human level reasoning capabilities. It doesn't have that. It's generating word by word, sequence to sequence. It doesn't understand what it is generating.
[00:08:09] host: That's a good point, because what I'VE learned in the last probably year or so is that it's really taught me how to ask the right questions. If you're good at asking the right questions, it can really help you. But it's not asking the questions, you know, you really have to give it those questions. And I'm curious, you have a lot of diverse background. You worked at major companies like Apple, Cisco, Walgreens. What lessons from the corporate AI in that strategy do you think that colleges and universities can apply?
[00:08:43] guest: So as far as that, I wouldn't mimic the corporate structure because they have almost infinite resources compared to like if you compare the higher ed. I worked at a university, I know what goes on there. I mean it's like the first I'll say don't build infrastructure, okay and have a proper strategy in place. I mean Arizona State University is an outlier but most universities, especially small colleges, they don't really have an AI strategy. Start with an AI strategy. Marketing is a huge thing actually sales, lead generation have an AI strategy. Like strategy is more important because resource allocation becomes important because at corporations, every corporations I work with, no one builds their own thing. Yes, Apple owns its own pipeline, Walgreens has its own pipeline for certain things. But certain things they don't specialize in, they always buy. So build versus buy assessment needs to be done. That should be part of the core strategy. Like what technology can we build in house cheaply and what should we really buy? What softwares, what AI software should I buy? Should I buy? Cursor, Any software development company, it's a no brainer. They should pay for the cursor, the team version. But it's not a one stop solution. If I need an agentic framework, Cursor cannot help. Like if I need a sale, let's say I need to create a sales funnel and I have disparate different sources of leads coming from different sources and all they get dumped into like some Amazon S3 bucket and we need to connect all those things. There is no solution for that in the market. We have to build one from scratch. In fact I know a startup which is working on that. They are connecting those different sales funnels into one integrated pipeline. Whether HubSpot Wall CRMs comes into and they'll integrate into the single pipeline. And they actually asked me to design it. We are in the process of talking to them and see how it can be designed. It's a very difficult pipeline. But that being said, in higher eds, especially because you're interested in higher ed, some of the solutions are too expensive. For them. And some of the solutions really don't make sense. So they really need to have a strategy or someone who is a consultant who knows the landscape. I mean you or someone who can advise them, they need to build that strategy for those. And here is my build versus buy assessment. This cost $250,000 and your ROI is over 25% in the first year. So that thing should be copied from the corporate architecture, not their actual architecture, because corporations spend millions and billions of dollars on building their pipeline because they're operating at scale. I mean, the cost is almost zero when you operate at scale. But for universities, they cannot operate at that scale. I mean the scale doesn't exist.
[00:11:33] host: What I heard you early on there say really resonated with me from what I've seen is make sure you plan, have a strategy. And I think that's where higher ed often falls short is they, they'll buy these tools but not have a great plan to roll it out or the resources. And so that's, I think that's a great takeaway. And then also for the higher ed audience, trying to make sense of all the AI, what are some practical ways you think they can start using it now?
[00:12:01] guest: Higher ed? Yeah, personalization is a big topic, but it's also a very thorny topic because data privacy laws are there. And also how much personalization can we do before it crosses that line and invades someone's privacy? Because right now we have enough AI tools to really clone our personality completely. We can create someone's digital clone if they give us enough data. Is higher ed willing to cross the line and create digital clones of the students? Like I said, don't because you can. Probably they don't. Students probably at that age don't care. Most of the students in that age group, they don't care what happens to their data. And that's concerning because even though they don't care and willingly give up their data, that comes back to bite them 10 years, 15 years later when they're in corporations. And we have seen that happen. And that's one of the discussions that's happening with TikTok because people are saying older generation is saying that giving up this data is bad, but younger population don't care. It's like, but then that data can be used by corporations which are based somewhere else and that can be exploited by some, not even foreign countries, just some corporations who don't have the best interests of the public, of US public. So that's one of the discussions that's the bigger Discussion. But when it comes to higher ed, I think the main thing higher ed should focus on if they really want to go to the personalization because personalization is big thing. Even Bill Gates says that, that if we do personalized education because different people have different learning modalities. And as a teacher, as an ex teacher, I understand that I used to teach 220 students at one time in a class at Arizona State University. I mean it's difficult to teach a class of that size mostly because most people don't learn the same way. I mean the teaching modalities has changed even with online learning. Some students can absorb by seeing some students like me, I need to work out the examples. So that's where the innovation should come, that higher ed should focus on creating that learning experience not personalized based on the student data. Like see what the preferred teaching modalities are and then create that tech stack that those slide decks which caters to that teaching modality. And don't go into too much into personalization because again, data privacy laws, ethical concerns, plus compliance requirements, the cost itself of compliance is so high and it's going to be even higher after the EU regulations are going to hit that it's just not a long term feasible solution.
[00:14:38] host: You made me think about.
I mean it's moving so fast. This feels like it was a year ago, about two months ago. We all will remember. Well, some of us remember what what happened with AI from a Chinese hedge fund that spun out a new LLM. Deep Seat. You maybe we said privacy made me think about this kind of data battle we might have with China when Deep SEQ came out.
I'll back up a little bit too. Last year I was talking to some Ivy League institutions and they were struggling with like what LLM are we going to use. And so now we have another LLM to consider. In short, right, we have the American labs, we now have this have Deep seq. They're American, maybe some others will come out. But really there's a handful of solid ones we're all aware of. Tell me how university should look at choosing what LLM and what maybe might be the downsides with deep seq vs OpenAI or gemini or one of those.
[00:15:37] guest: Yeah. So there are multiple matrix. You have to create the decision matrix. And I think you're very familiar with the decision matrix because you do that day in and day out. So in the decision matrix we have to have that matrix where what are our concerns? Cost of course is one of them. Privacy is one of them. And then there is a long term solution where how well can we maintain the technology? For example, let's say I'm in university, right? Or let's say take a better example, let's say I'm a small time private college and I don't have that much resource. I have like $250,000 extra left over to implement strategy and I need to resource allocate in marketing, sales and of course AI to enhance my retention policies and improve my lifetime value of the students or student cohorts. So I have to see my roi. So if I invest like a hundred thousand dollars in AI, how much return I'm getting? Like if I for $100,000, if I'm retaining at least $200,000 in student lifetime value, then of course I should invest $100,000. Now how should I invest that $100,000? Now I have to see that for $100,000 means I have two options. I can host a large language model locally, fully private, fine tune it on my student data. Highly private. But the upfront cost of fine tuning that large language model would be close to 15 to $25,000 in fine tuning and then operating on on Prem. If you don't want on Prem you can do on AWS GCP. Running it for a year. Yeah, comes around for 100,000. A self hosted large language model can be done. But what if I don't have the budget for $100,000 a year? I cannot fine tune and self host that large language model. In that case my only option is those cloud hosted platforms like OpenAI, DeepSeek. Now the second step comes in which is how concerned am I about privacy, what regulations am I under? Which state? I'm if I'm in eu, I'm under EU regulations. If I'm in us I'm not under any regulations. But there is concern that my students data will go to China and I really don't want that. So my only option is to either use Claude or OpenAI or Groq has their own self hosted llama, three versions that they're running which are quite cheap. So that might be one. But the problem with Groq is that then you have to hire an engineer and pay that person $80,000 a year just to get it set up because it's not easy to set that up.
[00:18:07] host: Are the open source ones harder to set up like Llama or Deep Seq?
[00:18:10] guest: Yes, deepseek and Llama both are open source and llama you can the open code and open source so you can have LLAMA download Llama and use it. I'm forgetting if Deepseek the full model is available yet. I think the full model is available, but I don't remember recall seeing it on hugging face. You have to verify that.
[00:18:29] host: I think it is too, but we'd have to check.
[00:18:31] guest: Yeah, we have to check. Yeah. For llama 3, it's free. If the 405 billion model, it's I think it's around 600 gigabytes or something like that last it's 636 gigabytes for self hosting and you can do it on Groq. I mean the upfront cost is high, but if you're going to use it forever. But the problem is AI is moving so fast, that model will get obsolete within a year. That's why most even enterprises don't self host anymore. They were self hosting before, but they stopped self hosting. They started using those OpenAI API calls. But OpenAI is going to raise their prices pretty soon. So what are you going to do then?
[00:19:07] host: So yeah, I have a theory that Google is going to just keep passing everybody up or at least stay competitive for a long time. OpenAI, I feel like has a head start. They just launched it to the world, but they didn't have 100 million users.
[00:19:20] host: Right.
[00:19:20] host: They came from nowhere. So all these other companies like Meta and Microsoft and Google have hundreds of million users and they've almost, you know, caught up already. And there's, there's boards where we can kind of compare all these. Google has access to all of our Gmail and our calendars and our YouTube and I'm already seeing their text to video is almost surpassed, you know, all the other tools. So I'm, I'm thinking Google is going to be really a player for a while.
[00:19:46] host: Especially for the schools that are on.
[00:19:47] host: Google, they just feel like they have a huge advantage. Is that, am I off track here or is that like are they on an equal playing field really? Or these bigger companies have an advantage?
[00:19:57] guest: No, you're right, Jeff. Google will take over soon because Google has the best research team. It's even better than the Facebook AI research team, even OpenAI research team and they pay them handsomely. I know a few of the people in Google DeepMind research team, they're the best. The reason Google is slow is because they're doing it right.
[00:20:17] host: Yeah.
[00:20:17] guest: And they're making sure those models are done right, trained right, follows all regulations. Because Google has like heavy scrutiny, they have no rush to run to the market. I think Google's model is that they don't want to be the first, but they want to be the last who completely takes over the market. They did the same thing with the search engine space. They took over the market. They were not the first mover, but they are the last mover. I think that's what they want to do.
[00:20:41] host: They do it over. I remember Google Chrome came out in 2008 and I was like another browser. Who needs another browser? That was just 2008 and I remember it and it was took over the world. So like that's, that's what I'm seeing too. I use Google AI products less than the rest just because I'm already used to, you know, OpenAI. But you know, from what I've seen, it's, they're right there.
[00:21:02] guest: Yeah, Google models are the best. And on top of it, Google has their own proprietary hardware so they're not really dependent on Nvidia. So they can run their models on their proprietary hardware. And that's why Google cloud platform is 10 times cheaper than AWS. It's harder to set up. You'll not find many engineers who knows GCP, but it costs 10 times less. So what will cost you $25 like a full P something on AWS GPU model with Llama Host and everything might cost you $25 an hour. It will cost 2 1/2 dollars to host on Google Cloud. Like just an example, there's a lot.
[00:21:40] host: Of concern about AI replacing jobs. I hear it all the time. How should higher ed, specifically communicators, marketers, how should they think about AI as a collaborator rather than competitor? Or what's your take on the job replacement power of AI?
[00:21:57] guest: Yeah, you already said it. I mean that's the usual thing, right? I mean AI would not replace jobs, but the people who are using AI will replace the jobs.
I will take the jobs from people who are not using AI. Like this has always been. It will replace some jobs like really low hanging fruits. They will be gone. For example, there is already autonomous lawnmowers. I have seen a few of them in the golf courses here in Arizona. So there are autonomous lawnmowers already out there. So guess what that does to the landscapers. Many of them will not have jobs. I mean, for example maids, we have robo vacuums. Maid service is gone. Like in India, the maid service was a big thing. Most of the maid service is gone because robo vacuums are taken over in AI applications. Copywriters like you probably don't hire any as many copywriters that used to do before you can write most of your own copies nowadays.
[00:22:52] host: Let me give you this example and it's not really marketing related, it's a little more engineer. Y'know my son is about to graduate from college with a data science and statistics and a linguistics double major. He's done it real quick like three years, but it's coincided over this AI revolution. Like when he went in, I wasn't Even really on AI. It was before ChatGPT went public, you know, was out. I was super excited with what he chose for a major and just in this past six months or so I've become more nervous for him like a data science major. Three years ago I was like, oh my gosh, he's going to be set, he can work anywhere. And I still believe that to some extent. But if you're a senior level data scientist right now, are you really going to be bringing on junior level data scientists when AI is here? So that's, I'm fearful for him be able to find a good job.
[00:23:42] guest: You know, he's a data scientist, right? So I was talking about data analysts. The data analyst jobs would be automated and it's already automated by Power BI and power apps. Like it's already integrated into Microsoft. So data analysts, those jobs are gone. Business analysts, data analysts, Some people are still hiring business analysts, data analysts. But we mostly gone. But even if you look at that, there will be some, still some residual demand for that because you still need someone to enter the data and do the job. Right. Like we can automate using voice text. But if you're a CEO and you have to have your market demographics analyzed and segmented, you can say that to an AI. I mean what is the guarantee that AI is going to follow through? Like someone needs to do those things. Like if you have to run marketing campaigns, you have to do ideations and all those things. Like you need a team, some people are doing AI agentic teams. But it's not going to work. Like you still need one human being who is orchestrating that. So you need a human orchestrator. So might be 80% job reduction. That's what we're seeing for the really low level job. But not for data scientists. Data scientist is much higher level like data analysts, those jobs, those are really like click and drop, doing Excel spreadsheets. I mean those things are pretty much automated right now.
[00:25:00] host: And I think it's going to be a gradual thing. It's not like it's going to be like mass layoffs that like oh, AI is replacing jobs. It's more like they're not going to hire as fast because these current teams are going to be getting more done and you might be compared to your peers a little more. So if you have peers that are using AI really well, you better keep up with them. But you probably won't. You know, that's the way I think it's going to happen is like and you if someone leaves, they might not replace that person as quick or at all.
[00:25:24] guest: Yeah, that's exactly what's going to happen. If someone leaves the job, they better be very sure that they have a job lined up, otherwise they're not going to get back that job. I mean it's usual for all enterprises, all corporations. Like they usually either eliminate the job or outsource it. I mean right now they're outsourcing to AI.
That's what's going to happen to all jobs that are going because anyone quits a job and cannot find another job, their job would be given to AI. That's what's going to happen.
[00:25:51] host: So on your podcast and I'm going to put a link to this in the show, notes Machine Learning Made Simple. You break down AI trends. What's the most surprising AI development you've.
[00:26:01] host: Uncovered in one of your podcasts recently?
[00:26:03] guest: There are a couple I'll talk about the last one I just did. This is a new breed of large language models called diffusion large language model. This is coming from the text to image generation models. So the diffusion models are essentially text image generation. You start with pure noise and you blend in your textual latent information and you generate and essentially guide your generation process using that textual information. So till now that cannot be imported into the transformer architecture because there are some issues. So they cracked the mathematical code that allows it to happen. Till now the main issue was that transformer based models were not time based stable diffusion. Usually they have time steps from time step one to ten. Like for example each step you inject your prompt and slowly guide it to the let's say a cat playing a ukulele. You slowly have to guide it like intermediate picture doesn't look like a cat, so you're prompt to slowly nudge it towards looking a cat. So similar process wouldn't be done for Transformers because Transformers is usually they have a two by two matrix known as the self attention matrix, which doesn't allow that. So they broke through the theoretical barrier and showed this new paper showed that it can be done. So that was a major theoretical breakthrough. Now the transformer models can be totally replaced by these Diffusion models. And if you look at the generation speed and if you ask, let's say GPT to write a piece of code, even Claude, it will take some time to generate the code. Like let's say 5 seconds, 10 seconds. Depends on how big your code is. If you ask the diffusion model, it does an instantly at a blink of an eye, like a microsecond. It's like it just appears. It's just like it has appears. Like for example, if I ask you what is the color of sky, your answer would be immediate, right? Blue. Because you have a latent understanding the color of the sky, you can immediately give the answer. The only delay is in the processing of understanding and responding. That's the delay we are seeing because now we can generate code almost instantly. If this is taken up and not killed by OpenAI and other big corporations, this can become the norm that we can generate almost instantly. We don't have to wait.
[00:28:11] host: So that's going to help support agentic AI, Right? Because what we need is like speed at scale, right?
[00:28:16] guest: We need speed. Yes.
[00:28:17] host: Right.
[00:28:18] host: Because you look at when ChatGPT came out two years ago or two and a half years ago, you know, we would see the line of code be written. It's like you probably almost typed that fast. It was incredible that it was like coming out in as fast as it was, but the benchmark has just increased so rapidly. Yeah, it's incredible. It made me think of one thing too. I've been doing a little research on Y Combinator. You know the big accelerator that's launched Airbnb and Doordash and Stripe. So they have their winter class and I know a guy who's in it, he got into it. You should check it out. I'll send you a link to this company's called Mastra. But I looked at who are these companies that are being selected for Y Combinator because it's like, it's like a 3% acceptance rate or something. Right. It's hard to get into Y Combinator. 75% of the Y Combinator class, this latest class, are AI driven. You know, even the ones I just made, examples I made Airbnb and Doordash and Stripe. Those weren't AI, you know, that was.
[00:29:17] host: That was years ago.
[00:29:18] host: Right. And they probably they've integrated AI. But it's crazy how fast everything is going towards. Most of those, I don't know what the number is yet, are agentic, some sort of agentic, you know, AI companies.
Hopefully. I have a blog about it pretty soon.
[00:29:33] guest: Yeah, that's awesome.
[00:29:34] host: I want to wrap it up and ask you one piece of advice or to give a piece of advice to let's say you're a university leader, you want to start implementing AI. What would you piece of advice would you give people?
[00:29:47] guest: Have an AI strategy in place?
[00:29:49] host: Yeah, that's a good one. All right, well, I'm going to let you go. Sohoto and I will put links to your podcast and your LinkedIn in the show notes. And it was really great talking to you on the show.
[00:30:03] guest: Yeah, thank you, Jeff. Thanks for inviting me. It was great talking to you as well.
[00:30:07] host: Okay, bye. Bye.
[00:30:12] host: As we wrap up this episode, remember EdTech Connects is your trusted companion on your journey to enhance education through technology. Whether you're looking to spark student engagement, refine edtech implementation strategies, or stay ahead of the curve in emerging technologies, EdTech Connect brings you the insights you need. Be sure to subscribe on your favorite podcast platform so you never miss an.
[00:30:35] host: Inspiring and informative episode.
[00:30:37] host: And while you're there, please leave us a review. Your favorite feedback fuels us to keep bringing you valuable content. For even more resources and connections, head over to edtechconnect.com your hub for edtech reviews, trends and solutions. Until next time, thanks for tuning in.
