⚡️ The State of AI Engineer Hiring: Cheating, AI Adoption,Junior Devs — Vivek Ravisankar, HackerRank

Okay. So here we are in the Linspace studio with Vivek Ravisankar of HackerRank. Welcome.

Hey, hey. Thank you, thank you for having me on the podcast. How are you?

Uh, good. I, I mean, I am, uh, here in Hawaii. I'm, uh, on a offsite, but, you know, the show must go on. And, uh, we've been trying to schedule this podcast for a while, so I, I was like, you know, "I'm not gonna make you reschedule again," like you're a busy guy, you got lots of things to do.

But like, um, uh, you know, like I think the state of hiring is always like very interesting. People are cheating at exams, and then people are using, uh, AI to also like evaluate candidates. It's just a mess. So, um, I guess like what's the state of like engineer hiring these days?

What, what are you seeing?

Wow. Yeah. Let's, let's start right there. Hey, first off, love your commitment. You're on a vacation, looks like in Hawaii or wherever you are-

Offsite. Offsite

... but you're still, um- Offsite. Offsite.

The cognition company offsite. Yeah.

All right. Okay. So then you're working, um, but still love your commitment on this. What's the, what's the state of hiring? The state of hiring is in a very, very interesting, interesting spot. So maybe I could just like take a couple of dimensions in terms of, uh, in terms of looking at it.

So I would say if I have to think about the overall jobs perspective, I think Indeed, uh, In-Indeed has this... Indeed.com is the number one job site. Reasonable barometer for developer jobs. Not I wouldn't say it's like the perfect one, but if you look at the, if you look at the job board index, you would find that the developer jobs index has flattened year over year '25.

Actually, I do have some stats. Do, do you want me to share?

Go ahead.

All right. Hey, so, so this is the Indeed's tech job posting index. Um, again, Indeed is the number one job site, reasonable barometer for developer jobs, I would say. The blue or the purple is the, is the one for tech, and the green is, is the overall.

You could see, I think they reset the index to 100 in '20. So you could see a very, very sharp rise in 2021, uh, early 2022. This is the Zip era. I think people were hiring like crazy. And then you can see like the GetFit era, it declined pretty rapidly.

So, so I think the amplitude of the w- swing of tech jobs is way higher than the overall one. But if you zoom in a little bit, you would actually find that 2025 job index is flattening relative to 2024.

So the decline has stopped, but it's nowhere close to the peak, nowhere close to 2021, n-not even like close to 2020. But on the other hand, if I were to just like take a look at the AI specific jobs, this is from, uh, this is from data from a, from a website called TrueUp, which, which does a pretty good job in like AI, uh, jobs and, and others.

If you actually take a look at it, the AI jobs tell a completely different story. Uh, this is all of the AI jobs that are posted on the site, uh, very heavily focused in, in, in Bay Area, uh, tell a, tell a very, very different story.

Then there's an-another analysis done by, uh, I'm blanking on the name of the, uh... This is like a v- data driven VC firm, I think SignalFire. Yeah.

Yeah.

They did a pretty, pretty good analysis on this one, where this is another dimension of, hey, what is happening in the seniority of hires? And what you're starting to see is the big tech companies as well as startups tending to hire more towards senior compared to, compared to new grads.

This is like probably broad based things of what we are seeing in the market, which is, hey, the overall tech jobs is the decline has stopped, which is good. The AI jobs are on a rise, and companies starting to move more and more towards like senior hires.

That's what the market data says. One of the interesting aspects that we have is, I mean, like we work with, like we work with close to 2,500 companies across the globe, across sizes. Um, this is all the way from a seed stage company to Fortune 500 organization.

So we see a lot of interesting data. And we're starting to see companies hiring more aggressively in both AI and probably the traditional application developer roles. I think the slope is starting to shift. We're also starting to see companies hire much more on early talent and new grads.

So we would see a very different chart. I predict that you're going to see a very different chart on both the slope of the hiring curve as well as the demographic of the type of hires in the next six-ish months.

Oh, what happened? Why-- Everything you said until that point was expected, and then this is new.

Yeah.

Like why-- Things-- It seems like the trend is reversing.

Yeah.

Maybe it's good news, but if we don't have any explanation for it, I'm, I'm a bit wary, you know.

Yeah, yeah, yeah. Uh, I have to say something controversial if I'm going on a podcast, right? Or something new. So-

Okay

... uh, so this, this, this may be. Uh, so I'll, I'll talk about this. We talk to a lot of customers. I talk to a lot of customers. This is everything from OpenAI, NVIDIA, Amazon, Salesforce, all of these.

Uh, we, we work with all of the, all of the customers. Um, I think over the last maybe 12 to 18 months, there was this notion of, hey, AI is just going to continue to get better at programming, which it is, or coding or software development.

And most of the work that junior developers do is essentially just like coding, so we're just gonna have AI do the job, and, and you're just gonna hire only senior people. And now people are starting to realize two things.

One is, while you always need a human in the loop, AI is really good, but like you certainly need human in the loop. There's a lot more work that you need. And second, if you really want to become AI native as an organization, I mean, guess, guess who-- Guess the people who are actually AI native.

They are like new grads. They are people actually coming out of, out of colleges. And so if you want to change the culture of your organization, you need to infuse the talent. You cannot-- It's, it's actually a little bit of a boneheaded move to like, you know, completely stop hiring people there.

And so I think like a lot of- Companies are starting to realize that, and that's a shift that we're st- w-we're seeing. Now, again, it's not bounced back to where it was, but the slope is changing, and, and I'm, and I'm actually quite bullish that you're gonna see a much different trajectory for new grad hiring over the next six to nine months.

Yeah. One of the most consistent things I hear from, like, OpenAI and, like, the, the coders inside the big labs is that the existing coders don't change their mindset about how to use coding agents as quickly as the people who are-- as the, the Gen Z interns that they hire who come in and like-

Yeah

... just, just like, you know, trust the model and... Most of the time it fails, but like 25% of the time it works, and it works in a surprising ways.

Yeah.

And then they are like, "Oh, like, let's, let's, uh, double down on that." And that's, that's very interesting to me.

Yeah. Yeah, 100%. Uh, I mean, like I, I would actually say the... Even, even in our own company, it's been way easier to get new grads and early talent just, like, be much more AI native than I have to, like, push really hard on the senior engineers and others.

Like, "Guys," like, "why aren't we using this?" Yeah, we're, we're starting to see that come up more and more. And there are like... And, and, and the new grads who are coming up, they, they're not like, you know, "Okay, I just, like, graduate.

I just took these courses. I got a 4.0 GPA and I'm, and I need, need a job." They are like real 10X people who can, like, shift the trajectory of your organization. We have a few of them as well.

I'm not gonna name names because you're gonna hire them, but like, you know, we, we, we, we have some, some of them in our company as well.

Okay. So l- I skipped over introducing HackerRank. I think, uh, a lot of people have heard of you, but maybe, like, maybe not, like, the most recent version of you. What are the surface areas of things that you built?

Yeah.

All I know is interview platform. You have a community. You have, like, presumably like an employer side that like... And then I also saw on, like, your website you have some kind of internal skills discovery ranking type thing.

So what, what are all the things you have ?

Yeah. Yeah. Yeah. Maybe we should have started here. That's okay. Maybe you can edit this.

No, no, no. It's, uh, I like, I like it starting at the top and then, and then, you know, we'll weave in things as they come up.

Yeah. So yeah, may-maybe I'll give a little bit of background. I'm, I'm, I'm Vivek, one of the founders and CEO of HackerRank. Started the company about 12 years ago. I grew up mostly in India, and I moved here to start HackerRank, and we were the, um, first Indian-based company to get into Y Combinator.

I really wanted to be in the Bay Area. I really wanted to get into YC. Uh, so prior to, prior to starting the company, I used to work at Amazon in India as a developer, but I used to interview a lot of people and found that resumes didn't have a great correlation to your skills.

I'd interviewed people who were really good on paper but completely bombed in the interview and vice versa, and that was the motivation for us to start the company. So I started this company with my co-founder, Hari, who I went with, uh, who, who I studied together with in college.

And, uh, he's also one of the, one of the, one of the, uh, one of the guys who, who has quite, like, very low GPA, but like the best hacker in, in town, right? Like, and so he'll always, like, get rejected by all of these stupid application filters that would just, like, filter by your GPA and all of that.

So we started with the notion that skills matter more than pedigree. That was the core ethos of the company. That is still the core ethos of the organization. And the first product that we started was like take-home assessments, which is instead of looking at a resume, companies could actually create their own take-home assessment and give it to candidates, candidates can solve that, and depending upon how well they solve, they can decide if they wanna move forward or not.

This was a much better way of showcasing skills over pedigree. And then we realized that there is an aspect of using this ethos across the entire life cycle journey of a developer, which is in terms of how you prepare for a job, how you get hired, and how you upskill yourself within the organization.

So today we have five products that span across the entire life cycle journey. We have a community that's now over 28 million developers, where developers, it's free. I mean, there is a paid version, but it's predominantly free, where you can come in, learn, practice, hone your skills, earn badges, and, and others.

We have a product called Engage, which our customers host hackathons in our community to attract developers. We have Screen, which is the take-home assessments, which is where we started. We have Interview, which is a pair programming interview session that you can do, and Skill Up, which, uh, helps y-your existing engineers to upskill themselves to be the next-gen developers.

So these are the five products. We have, um, close to tw- uh, between 2,500 to 3,000 paying customers, 28 million developers, and, um, yeah, we're, we're, we're, we're, we're continuing to grow and, uh, we see, we see a lot of data.

I mean, that, that is a, a humongous amount of data. So, okay, what are your strong beliefs now on what is next engineer versus a regular engineer? Like, uh, how do you detect for that?

Yeah, no, it's a great question. I think here is, here is how we, we think about the next-gen developer, and we distill it down to four attributes. The first one is you need to be very strong on the fundamentals of software engineering.

Yes, AI is gonna do a bunch of things, but you still need to know the fundamentals. I mean, just like the way we think about it is, you know, your 90% of your flight is flown by AI, but you still need the pilot to know aeronautical engineering.

You wouldn't jump on a plane if your pilot is a wide pilot, right? Like, I'm sure, like, you'll get off. So you still need to know the fundamentals of software engineering. And that doesn't mean, like, you know, trick questions and things of those lines.

You need to know code complexity. You need to take a look at the code and say whether it's O of N or O of N squared. You need to know the basic data structures. You need to know the building blocks and all of those things.

Second attribute is you need to be able to use AI across all aspects of SDLC. Well, that was supposed to be the job of a developer, but we got so bogged down, uh, with just, like, writing code and all of those things, and we started to have, like, specializations across each of them.

So I think, like, you should be ab-- Now that AI agents are getting better in each of the aspects, you should be able to work across the SDLC using AI. The third attribute is every new technology comes with a set of concepts that you need to, like, learn, and specifically, so you need to have a very strong knowledge of AI concepts, and this is everything from prompt engineering, to fine-tuning models, to setting up MCPs.

And like there's just gonna be more and more of this that's gonna come up. And finally You need to have a lot of taste and good taste and business acumen whenever it comes to building products. So this is how we define the next-gen developer or the 10X developer.

You're strong in fundamentals of software engineering. You can use AI across all phases of SDLC. You have very good knowledge of AI concepts, and you build with good taste and business acumen. And this is how we've structured our company.

This is how we're structuring our products to help developers become the next-gen developers, to help companies hire the next-gen developers, and to upskill their existing people to become the next-gen developers across these attributes.

Yeah. I mean, I think that's-- I mean, the, the AI piece must be new, right? Like, how did you, you know, develop, develop this over the last three years?

Yeah, yeah. It is new. So, um, so one thing that we, we, we realized and we did was our goal has always been like, "Hey, whenever it comes to evaluating a developer, we need to make sure that it can mirror your daily job, the day-to-day job."

And if you think about the evolution of sort of technical interviews in general, I think, I guess, like maybe in the 1990s, people used to ask questions like, "How many golf balls fit in a 747?" Um, you know, think, things on those lines, like to, to just like evaluate your computational thinking and stuff.

It has got nothing to do with, uh, programming. And then, and then we started to, uh, ask people to write code on a piece of paper. Uh, for what it's worth, I got my job at Amazon, this was like, uh, certainly I'm, I'm dating myself over a decade of writing code on a piece of paper.

And then, and then we shifted, uh, but it was still like data structures and algorithms, and then we shifted the same writing code on a piece of paper to online. Then we started doing type projects. And now where we see companies going is towards evaluating them on real-world tasks on code repositories.

That's where, like, the world is going. And when you, when you think about, like, evaluating real-world tasks on code repositories, then your environment, which is the IDE that you use and all of those things, also need to mimic the real world.

So we took a lot of inspiration from the Cursors of the world or the Windsurfs of the world, what- whatever it is like in terms of like IDE, Copilot next to it, building code repositories, and making this as real as possible.

So that's been the evolution for us. Uh, so we've always believed in it. We also think the next evolution is gonna be beyond code repositories, um, because again, as much as, as much as IDE and AI assistant and code repositories like the home for, for, for developers, that's not the only place that they work in.

It's always like the prod is down. Uh, okay, so you go to, uh, Sentry, you look at logs, you look at a bu- bu- bunch of things, then you go back to your IDE, you try to fix. So the next evolution of the interview might be, might even be beyond, uh, just like a code repository set of tasks, uh, with an IDE.

So we think the, uh... We're pushing the bar, we're pushing the industry to make your interview process mirror your real-world work as much as possible.

Anything on coding agents, like the Cloud Codes and all that, or I guess copilots or I don't know what the, what era we're in.

Yeah, yeah. Um, well, you must be knowing more than me. You're, you know, close to it, but-

This is my whole life right now. Yeah, no, like, not only do I work at Cognition and Windsurf, and then I also, you know, am, like, organizing this AIE code thing, which, uh, you know, you're, you're definitely invited to.

Yeah, yeah.

But, like, it is, like... Yeah. Everyone's launching a terminal, and everyone has, like, their cloud thing, Codex launch to cloud thing, Cloud Code launch to cloud thing, and, like, uh, I don't know if it's, like, part of the skill set to, like, try everything or just pick one.

The way we think about that is, um, I mean, like, uh, the way we think about that and, and we, we define our hiring process also like that is, um, there are two major work types of workflows in, in, in today's world that developers use.

One is you're going to tell an agent or actually a bunch of agents, "Go do some stuff, and, uh, once it's done, you're going to review the PR." That is one, one workflow. The second workflow is you're gonna use AI as a copilot to actually do the task.

These are, like, the two workflows. So the way we test for the usage of agent stuff is less about can you prompt the agent c- to do the work, but more in terms of, hey, I've actually generated a bunch of PRs from Devin or Claude Code or Gemini or like Codex or whatever.

Can you go ahead and, like, review, and how good is your reviewing capabilities on that? And then, of course, on the AI as a copilot or an assistant, it is, "Here is a code repository. Here is a, here is a feature that you need to build.

Go build it alongside Copilot." Of course, you can also use an agent, but, like, you know, that's, that's, that's kind of how... But those are the two predominant modes, uh, that we see. It's also likely the reason why Cognition also believes that way, right?

Like, you guys acquired Windsurf, which is more of an IDE and a copilot, as well as, like, Devin is more of an agent. I think, I think we're gonna find these two as the predominant workflows. Uh, will one supersede another?

I don't know. We'll have to wait and see. But as of now, I see both of these workflows existing hand in hand.

Yeah. I, I, I think so too. Um, what I ca- I've been calling this, like, the sort of, uh, async value of death. Like, basically you want either, like, the, the sort of super local, you're working on the hardest problems, you need the, the, the agent to help the human, or the other side, where it's, like, super async.

You don't really care. It's, like, just changing a button, something that's super commoditized.

Yeah.

So you can just send it off to-

Yeah

... Devin or whatever.

Yeah, yeah, yeah. A- and I mean, like, even internally, we're, we're getting to, in our company, we're getting to a point of it's not just, like, developers, but solution engineers, support people being able to tag something like a Devin to say, like, "Hey, there's a typo in this error message.

Go, go fix it." I mean, like, you, you don't want to bo- bother engineers for that. It's, uh, it's always in the low-priority task, never gets done. So yeah, so we're, we're, we're, we're starting to move in that direction.

Yeah. It, it's really, like, it's great for the marketers, the PMs, the CEOs who, like, you know, should not own the code-

Yeah

... but at least can, like, prototype the PR-

Yeah, yeah

... and say-

The CEO shouldn't just, like, misuse and just, like, cause a bunch of tech debt which engineers have to go and fix, so I'm very, I'm very, uh, cognizant of that. Yeah.

Yeah. Well, so, you know, you have five products. Would you still, still say, like, the, the interview stuff is the, the core business?

Yeah, I would say the screen and the interview, the two of the five products is about, like, 80 to 85%. And then I'd say Skill Up is starting to be, like, close to, uh, 10-ish percent. Engage is 5%.

Community, we just introduced pricing plans, so it's a very, very small percentage.

Wait, how do you price Community?

Yeah. So we are-- So, so bunch of all of the challenges are free, and now we are introducing two plans. One is a Plus and the other is Infinity, where, uh, you could just have AI tutor for, uh, for all of the challenges where you can actually learn, uh, from it.

The Plus has limited, um, limited amount of, uh, tutor that you can use, and Infinity has, like, unlimited. You can just, like, use it for as much as you like. So that's, that's our, that's our pricing plan.

Okay, so you're, you're killing LeetCode, basically.

I guess Clueless can LeetCode.

Okay, okay. Let's, let's start, let's start the Clueless discussion, right? 'Cause you're, you're dancing around it. Um-

Yeah, yeah.

Yeah.

Yeah.

Clueless came on scene. Was there, was there a major AI cheating tool before Clueless, or was it Clueless that just drew a lot of attention to it?

In-interestingly, I mean, like, Roy and I used to chat, like, uh, uh, you know, when, when he launched this, this whole thing. Uh, I guess now he's become a big celebrity. Uh, he doesn't respond to my DMs much.

But anyway, um, uh, actually, Clueless is not the one. It's actually Interview Coder. It's just that he got popular with Clueless. Interview Coder is the one. Uh, were there other tools before that? I, I'm sure it was, but I think, like, uh, Interview Coder just, like, caught the wave and then there were, like, a bunch of other tools that, that started to, that started to come through.

Uh, by the way, it's not as invisible as, as the website claims.

Yeah.

Uh, especially on Mac and things, and there are, like, ways that you can detect. Uh, we can, we can talk, we can talk about that. Yeah. It's, it's not, it's not as clean as, as it claims. Yeah.

So like, okay. The, the-- Just for everyone who is, like, not as clued in as us, uh, the idea is that it's, like, an overlay on top of your screen while you're on Zoom with somebody. And one-

Yeah

... while you're talking, it's transcribing and it's trying to feed you answers, right?

Correct. Correct. Yeah. Yeah.

And then like-

That's a good description

... people, people complain, like, "Oh," like, you know, "I see the guys, like, looking at, looking at this corner of the screen, therefore it must be cheating." Maybe.

Yeah.

Maybe not. Maybe he's just, like, you know, on the spectrum. I don't know. But like...

Yeah. So let, let's, let's maybe talk about, uh, how-- Let's maybe talk about all the ways people cheat, okay? So I've done a PhD on this. So we'll, we'll talk. Or, or, or maybe, maybe I'll, maybe I'll, I'll phrase it in a slightly different way.

Talk about all the ways that, uh, impact the integrity in your hiring, uh, process.

Sure.

Okay. So I think, like, major-- I'll, I'll bucket it into three things. One is you leak the questions online and people actually see it and, and, you know, you, you know the answer beforehand. The second is you use these unauthorized tools, whether it's Interview Coder, Clueless.

There's one called UltraCode. Uh, there are bunch of, bunch of things that, that, that, that, that, that come up. The third is, uh, impersonation. I ask you to take the, uh, interview or the test on my behalf. So these are the three ways in terms of how it actually does.

On the, on the-- And let's talk about, like, you know, what is our stance. The way we define integrity is it's not about whether you use AI or not. It's about whether you follow the rules. So we, we think about, like, integrity when we think about, like, engi- When we think about building the integrity engine is, hey, how do we inf- how do we make it very clear to the candidate that these are the rules?

And how do we make sure that you can prevent the candidate from sort of, sort of going away from the rule? And third is, if the candidate does exhibit suspicious behavior, how do we actually flag that both to the candidate as well as to the company immediately?

I mean, that's, that's-- those are some of the principles that we follow when-whenever it comes to the ensuring integrity of the process. So question leak, we actually run scraping across a lot of the different websites, a lot of different forums to actually see whe-whether questions have gotten leaked.

Can you, um, uh, Chegg is the biggest, uh, offender. But I, I love Chegg, uh, as a company and things of the size. I know they're going through hard times, but, uh, rooting for, for them. I hope they come back.

Oh, no.

Uh, uh, but, but Chegg is, like, right up there. There are other forums, StudyX. Interestingly, LeetC- We, we actually don't see a lot of questions getting leaked on LeetCode. I mean, there, there, there are, but, but it's not, it's not in the top three or the top five.

So we actually run our scraping mechanisms across a lot of forums, a lot of them, a lot of these sites to pull them up. We then do DMCA. Sometimes they, sometimes they take it down. Actually, most of the times they do take it down.

They respect and take it down sometimes-

Do, do you do this for free or what? You're, you're, you're, you're running-

Um-

... your customers' questions on...

Yeah, customers' questions-

Okay

... our own created library questions as well, all of that. Yeah.

So as part of your service, you run DMCA takedowns?

Yeah, yeah.

Oh, okay.

Yeah, yeah. Yeah. We do that. Yeah. Uh, so what-

And they all, they all respect it?

We do that. Sorry?

They all respect it? They all respect the, the DMCA takedown?

Pretty much. I think, like, you know, I'd say like 95%-plus actually respect it. And we are also very cognizant of, you know, sometimes people actually upload this on, like, some GitHub repository and, um, and, you know, when you put a DMCA, like, you know, you want to shut down the entire GitHub repository.

We got caught once like that in Hacker News. Uh, boy, like, you know, you get, you get like that on Hacker News, it's just like, uh, it's, it's just like the giving like a very soft ball to, to a bunch of hungry lions.

Like, you know, they just like come and pounce you, "Oh, you guys suck, you guys suck," blah, blah, blah. So we, we have to like tweak and make sure that we're very confident that it is a leaked question and like, you know, when, when you're going DMCA, there's like a human in the loop before you actually submit the request and all of those things.

So we've done a lot of work in making sure... I mean, ultimately, ultimately, we're striving to make sure that we have like a fair process, um, you know. And as much as customers want it, it's also the candidates want it.

Like, I've seen like candidates DM me on LinkedIn and Twitter, uh, Li- or X saying like, "Hey, I want you guys to up your, up the bar in terms of how much you think about, like, caring about the hiring process and integrity, because I'm doing it in an honest way, but the guy next to me is not, and I don't want to like, you know, lose the opportunity."

So it's as much a big deal for the candidate as much as it's for the company. So that's the first pillar of like the question leakage. The second one is all the unauthorized tools that you use. So a few things that we do here.

One is, um, we have trained our own, uh, plagiarism, uh, model. Which can detect, uh, whether you're using an external tool to write code or not with about, like, 85 to 90% precision. So we have actually trained. So which means, like, whether it's, whether it's Clueless or any other mechanism that you actually use, there are some coding patterns, everything from, uh, how quickly you solve to how many times you edit certain things.

I'm sure, like, now people are watching this as though, like, "Oh, Vivek is open sourcing the algorithm. Let me try to, like, game this." Like, no, it's gonna be very hard for you guys to game it, but I'm okay to, like, you know, just, like, tell it out.

But, like, so we, we've trained our own, um, sort of, sort of plagiarism model to detect that. And on top of it, we have, like, varying degrees of, uh, protection that we give for our customers. Like, so we have introduced this thing called as a proctor mode, which, which asks you to share your screen.

Um, it can look for applications. It can take screenshots on a regular basis. We also have an app, um, which, which is the next level, proctor mode plus plus, uh, which now kind of acts as your operating system, so it can actually shut down apps, uh, like Clueless of the world, um, as well.

So we have, like, different mechanisms, uh, from right from basic plagiarism detection of coding patterns. I say basic in the sense of, like, you know, the first level all the way to, yeah, download an app and it can actually shut down other things.

The third one is impersonation. We do a lot of image verification. We also, like, suddenly you would find, like, two people coming on the frame. Suddenly you find a different, different person in the frame in the middle of the test.

So we, we see a, we see a lot of them, so we actually, like, you know, look at images. We, we parse through to make sure that it's the same person. So we do, we do all of this.

Um, I'll say one more thing. I know I've been on a big monologue. Um, above all of this, we are embracing AI. I think it may have been Steve Jobs, but I, but I don't know. It-- this might be one of the quotes that always gets attributed to somebody, uh, but they never said it, right?

Like, I, I don't ever think Henry Ford said, "If I ask people what they want, they would've... they want a faster horse."

So it's not him? Oh, shit.

I, I don't think Henry Ford said that. Uh, it's, it's widely attributed to him.

Clearly it's Albert Einstein said it.

Sorry?

I'm, I'm joking, I'm joking. You would think everything leads to... Yeah, yeah. Everything leads to Albert Einstein.

I missed that, I missed that. Yeah, yeah. Um, so I don't know, I don't know. This might, this may fall in one of those things. I think he was like, "Hey, the best way to embrace piracy is to make it official," or something on those lines.

And I think he was referring to Napster versus, versus the iTunes Store. You know, yeah, you're illegally downloading and stuff. Like, why don't I just, like, make it official for you to make-- for you to just, like, download it?

And so, hey, if you wanna prevent people from using all of these AI tools and stuff, why don't you just, like, give AI as part of your, part of your assessment-

Yeah

... as part of your hiring process?

Yeah.

So that's-- So yes, we're gonna do all of these things, but more importantly, at the top, we wanna make sure that, like, people get access to AI Assistant, which also means you also need to think about, like, a different kind of problem.

It cannot be the standard leetcode rote stuff that you wanna do, which will mean, so it's gonna be more real world, more code repositories. It's also something that developers really love and enjoy much more than, than the normal one.

So that's the direction in which, uh, we're, we're heading.

Yeah, amazing. I mean, I, I think that's, like, that's, like, really encouraging. I think that's, um, I guess enlightened or inevitable actually almost is, like, the best word for it. Like, you can't avoid it. Like, you know, you might as well get with the trend.

Yeah.

Yeah. I think, like, does it differ for more senior versus more junior now? Like, are you seeing that, um... I guess there's a lot of fear that there is a, there's a massive decline in junior hiring, uh, but we just talked about how junior might be bouncing back.

They may have more skills in the agentic coding. Um, yeah, any, any differences on seniority side?

So first off, I, this, it's bounce- it, it is the early signs are very promising, just so you know. Uh, because we look at how many people, uh, or companies sending assessments to junior versus senior, uh, how many people they are interviewing.

Grown about 20 to 25% year over year, um, relative to all of the other, other news, news items that you see on social media where it's like, hey, declining, everybody's gonna be out of jobs and stuff, and I actually expect it to continue.

In terms of the hiring process, yes, it's certainly different. I do think the fundamentals aspect remain the same, but, like, how deep you actually dig into, uh, varies a lot. I think for a senior it's much more around, like, system design and other aspects of it, so I do think, I do think that changes.

And, and I also think the more senior that you go, I think you, you have a bigger ego, so anytime you send a take-home assessment, you're probably not gonna, uh, not gonna attempt one. So I, I guess, like, you know, you reach a different kind of a stratosphere for, like, principal engineer and others where you're not even going to attempt a take-home assessment.

You're going to directly go into the interview. So from a rubric perspective, I would say, uh, new grads, um, fundamentals and, like, much more hands-on coding with AI Assistant. For senior, it is still fundamentals, but, but much more of code review and architecture and system design and, and other aspects of it.

And then, like, whatever, super senior or so, the principal engineer is mostly gonna be directly on interview. And yeah, and, and if you have the meta label, you, you gotta probably pay $5 million. So, uh, so, so, so that is there as well.

Uh, are you seeing, like, a lot of meta AI candidates because they just got laid off? I don't know if they actually could be.

We haven't, we haven't... Yeah. The, the, the, the, the recent one we haven't seen much. Uh, but I also think they're like AI researchers, uh, so we don't see a lot of AI researchers-

Yeah

... yet coming through. Actually don't, don't know how their interview process is. My guess is, like, there's such a dearth, they're just like, "Oh, you're an AI researcher? Come here. Here's, here's $1 million." I, I don't know if that's kind of, like, what is happening.

I think it's probably getting pretty crazy.

I've talked to some researchers. So Yi Tay is a friend of mine, and he's, uh, leading thinking at, uh, DeepMind. You know, mostly you can tell when you ask them training questions if they have the right intuition.

Mm-hmm.

If you're like, "What if I move this hyperparameter this way? What, what would happen to the model?" If they don't have an intuitive sense, if they're like, "I don't know. I have to check," probably they don't actually have the hands-on experience.

Interesting. Interesting.

And so, like, at, at some point, if you're really in ML... And, and there's like, you know- 200, 300 of these people in the world, they all know each other. Uh, so like, yeah, I, I mean, if, if you, if you know, if you've been through that kind of training or that, that kind of practical hands-on experience, then, then you know.

And, and like everyone else, you know, has, has more traditional ML work when it's not training a frontier model, and that's a, that's a different job market. And there's-

Yeah. Yeah

... there's also a market for that. I think, like, the, the interesting thing also I was, I was kinda curious, like, so, you know, some of your clients are, like, the, the big labs and, uh, I think like, you know, if there's a lot of people that wanna join the big labs.

You don't have to name anything in particular, but like, you know, what are they looking for? If I'm preparing for an interview, like, what do I do?

Um, I'm gonna say something, uh, I don't know if it's controversial or not.

Let's do it.

Um, the AI native or AI-forward companies

are the least AI forward when it comes to interviewing and hiring pro- process. I think Anthropic is not our customer yet, so I'm not gonna talk about all, all about our customers.

I don't want you to get in trouble.

I might get in trouble, but, uh, Anthropic has... If you go onto their Careers page, I don't know if they still have it. They had it, like, about a couple of months back. They make it very clear that you should not use AI when you're applying to, to the job.

Um, actually they had, uh, let me see if I can pull that out. They had this honor code. It's about a few months back, so I don't know if they still have. "Hey, while we encourage people to use AI assistants during their role, please do not use AI assistants during the application process.

We wanna understand your personal interest in Anthropic without mediation through an AI system, and we w- also want to evaluate your non-AI assisted communication skills." It's very interesting, right? In terms of, um,

in, in terms of, in terms of what, what they do. So, so, so what is, what is... Yeah, so that's probably an interesting one. The, the most AI forward companies, if I, if I think about it, are not as AI forward as you would expect in the hiring process.

The companies that are pushing AI hard are comes a lot from services organizations. These are very large services organizations, consulting services organizations. I think I can understand why, because it's this, it's like a margin business. You are selling human labor, and if you can actually make it more productive, it's actually good for the, for the team, for the company, for the organization.

They are actually much more AI forward and pushing through to get this. So that's what we see. Um, although, like e- I know every company is working towards becoming much more AI first, which is... And, and when I say AI first in the hiring process, it's going beyond the traditional algorithm data structure stuff to code repositories, having AI assistance built into it.

So yeah, so, so that's a shift. That's it's, it's much more early in the AI native versus the large Fortune 500 organizations or the large consulting kind of, kind of services organizations that we work with.

Yeah. Um, that's surprising, but maybe a- also the way, um, you know, they have a very high bar and, and, and, you know, they, they, they wanna make sure that you jump through the same hoops that they went through when they, uh, uh, they were hired.

Um, so excellent. Um, you know, like you don't have to focus too much on hiring. Uh, you know, I know you're, you're sort of very broad, general, you know, you have a lot of opinions on the, like, the, the sort of AI landscape.

Also, just kind of, like, stepping out of your role at HackerRank, like, what, what, what else is sort of like top of mind that, you know, that you really wanted to riff on?

There are a bunch of things that, that I, that I think about, that I, uh, uh, think through. One is why is, why is AI not as helpful as it, as, as it's projected at work for non-developer roles?

Um-

Okay

... um, it's, it's not, um... Yeah, so I- it's, it's always puzzling. It still doesn't create good presentation, still doesn't create good financial models, still writes, like, p- pretty poor docs. Even in the developer world, I think there's, like, mixed thing in terms of productivity, uh, gains.

Like, people say, "Oh my god," like, you know, this is... That- that's, that's what you get on social media, right? Like, "Oh, this would have taken me a week. I did it in two hours." Sure.

The, the consultant. Yeah.

Yeah.

Uh, yeah, I mean, especially the presentation one. Everyone seems to love the slop. Um, you know, there's all these startups that are, you know, putting out presentations and like... And it's like AI generated image that just, like, kind of doesn't mean anything and just, like, takes up space.

Yeah.

And I think, like, I think my, my, my learning from that is especially people want slop. People deal with all sorts of human originated swap- slop all day long. So-

Yeah

... replacing with AI slop doesn't really matter. And, you know, you, you, your standards are high. You think that, "Oh, I would never do, uh, AI slop presentations." But I tell you, like, there's millions of people out there who, like, their standards are not that high.

Yeah. You know, o- one, one interesting thing that I've observed is people... I mean, humans are just, like, wonderful. Uh, I mean, I, I have a, I have a newfound respect for humanity after all of the AI. We are adapting so quickly that now humans can easily, pretty easily distinguish whether something is done by AI or human, or at least, like, something is done entirely by AI or human.

I've seen, like, people react to, "Oh, this doc seems like it was written by AI." There was a guy who actually came and did a presentation for us, uh, as part of the interview. That was very obvious it was done by AI, but, but that's not, that's not the interesting thing.

The interesting thing is not only are people able to discern how much of AI was used or not, but if you used a lot more AI than human element, you start to ascribe a lower value to it. "Oh, this was actually mostly done by AI.

I'm not very impressed. Uh, what did you really do?" Um, and that is, that is starting to be an interesting aspect that we're, that we are figuring out. Um, so, so, so I actually think the more AI becomes powerful, the more valuable human craft and other elements are gonna become because people are gonna easily determine, "Oh, this is all done by AI," and just gonna attribute a lower value.

Yeah. Oh, yeah, I think it's interesting. Uh, I'm actually a little bit more open to it in the sense that if it's helpful, it doesn't really matter if it came from AI or humans. But-

Okay

... when the human submits it to you, they s- they sign off on the whole thing. It means they-

Yeah

... they, they, they stamped that they review it. So if I come across slop in there, I don't care if it's AI slop or human slop, I blame you, right? So you better come to me-

80%

... you better come to me with, like, good stuff. If your judgment is poor, whether or not you wrote the thing, I don't care. If your judgment is poor, if you cannot see, if you cannot review for me that, that you are-- you have f- like slop in there that should be taken out or edited by human hands, then you probably shouldn't work for me.

Yeah, but, uh, I mean, in the sense that can you use AI to... Can you use Midjourney to create beautiful images?

Yes.

Sure.

It's a different kind of brush.

Um-

Yeah, totally.

Yeah. But you can de-- you can... Or I don't know, maybe I can discern somebody who just said, "Go create a mountain," or something, versus like putting in the effort of like, "Hey, this style kind of painting," and like oil pastels or colors and things on those lines.

Oh, yeah?

There, there's like a diff-

Okay.

At least I can see it, or at least I've been spoiled by my, by our design, design team, where it's very evident that I would've just done some very high-level things versus like you're actually creating prompts that make sense.

Like, for example, like we're, we're in the process of even like evolving our brand and all of those things, and one of the things that we are leaning in is the idea of, I think human craft is going to start to get more and more valued as AI becomes more powerful, which is the irony.

And so all of our branding is going to be much more analog, much more about like blueprints and things which you would associate with humans. And even if you're using Midjourney, it would have this sort of oil painting kind of a feel or a texture to it because like, you know, humans like paint.

Um, so all of our branding styles and images need to showcase the human element in it. And so what are all the, what are all the things that humans are known for? Craft, like, you know, painting, blueprints, and analog stuff.

That's, that's what we're doing. And so it's important to not just generate any random image just because it looks good, but how do you actually generate with much more thought? Uh, so may-maybe, I mean, those are the things, right?

Like you're easily able to differentiate from somebody who actually used the tool to get what you want as opposed to just like, "Oh, I'm just gonna put a few prompts and see what comes out of it."

I think that is a really interesting art. Uh, we're actually thinking about creating a second show for Latent Space called High Taste Testers that where basically it's like a small-

Oh

... little competition, and one of them would be prompting image and video models on like getting good output. And then each-- It, it'll be filmed in a restaurant. Every course will have a different prompt, and then everyone just like competes.

Nice.

And we'll, we'll, uh, we'll rate it. I think it would be kind of fun. But yeah, that, that's cool. I, I mean, like, yeah, I, I, I think like the automation and knowledge work, you know, like the number one thing does have EMF, which is recording sales calls, which is Gong, which is, what's the other one?

Uh...

Oh, you think about Granola?

Granola, that's the one that-

Yeah, yeah

... that's sort of like the, the more like AI native one. But like they all do the same thing. Notion has it. But like I, you know, I think like AI for non-coding use cases, you know, coming back to the original topic, is starting to percolate out.

And, and I think it's just, it just takes a long time. I think Arc, uh, sorry, Atlas from OpenAI is like very interesting. It's a browser that does work for you. Obviously super limited, but like it, it is a huge progress since the first operator, which-

Yeah

... which was only in January.

Yeah.

So like-

Wait, the operator was only in January?

Yeah.

Wow. I thought it was like earlier than that. Wow.

Cloud Coda-

What, what do you think-

Cloud Coda was launched in March.

That's crazy. Yeah, yeah, yeah. Yeah. Our timelines are all getting compressed. What do, what do you think about Atlas?

Atlas, I like it. As far as OpenAI launches go, it is one of the better ones, 'cause a lot of them are like very vibe coded. This one is less so. But I am very spoiled by Arc and Dia, right?

Arc actually, uh-

Oh, yeah

... because of the tab groups and the vertical tabs and all that. And-

My God, I, I was about to say the same thing. Like I cannot believe they launched the Dia without the side tabs and, um, uh, without the functionality of side tabs. Like it's, it's prob- Still on Arc, by the way.

It's not agent and stuff only because it has the side tabs and tab, uh, uh, tab grouping.

Orion, I've tried Firefox, this is the best one, right? The, the problem is, uh, I think... So, uh, and, and then, you know, just to close the topic on Atlas, the head, the head of Atlas has tweeted about, oh, like people asked him like, "Can you please have the vertical tabs from Arc and the spaces and the tab groups and all that?"

And he's like, "Well, have you seen our side-scrolling tabs? Like we put a lot of work into that one." And I'm like, "Okay, this guy's never shipping it and he doesn't care." Uh, so like I think there's a difference between like the general popula- population, which only, you know, is just like...

Which I think that's what OpenAI is going for. And then the pro users, and the pro users we-we're just gonna use probably the browser company stuff, and that's not surprising. I think like what's interesting is actually what is Google gonna do?

Because now, oh, you're coming for Chrome, like screw you. Like I'm gonna change my license to ban you. Like probably they should at least do that. Um-

You know what's funny?

Yeah.

Google open, quote-unquote, "open source transformers," Google open source Chromium, and literally like, you know, tho-those are like the two technologies that all of these companies are being built on. Yeah.

But like, you know, this is open source as it should be, right? You know, like, okay, you know, like, ha ha, they like didn't, didn't like capitalize ama- amazingly, but they're still Google. They're still, they're still huge. They're still catching up, like, uh, you know, and like in some ways arguably number one on, on depending on what metrics you look at.

And they have like an enormous runway and like infinite money. Like I'm not crying for them. They open sourced it and it, and people are using open source as intended. This probably accelerates humanity because they made the decision to open source.

If they did not, we would be stuck in a Google monopoly for like the next 100 years.

Yep. Yeah. Yeah.

Um, so like we should just be grateful and not make fun of them because, because like, you know, they, they just have to catch up and, uh, you know, on their own merits instead of, uh, using lawyers, right?

Which I think like is the right way. Um-

Yeah

... yeah, I think, like, uh, the other thing I'm, I'm really interested in is, like, on the Arc/Dia side, the founder side, right? A lot of founders are going through AI native reinventions basically, right? You have a company pre-AI and then now, now everything is AI and, like, you're...

Now you have to decide, uh, whether or not you, like, have to sort of pivot the products, and that's exactly what the browser company did. They have Dia, they have Arc. Uh, and I think the problem with their customer base is the customers are forcing them back to the pre-AI version.

Version.

Right? They're like, "No, no, no, I don't want the new thing. Just give me the old thing." Uh, and that's very scary, uh, for a founder because you are trying to do a, a brave-

Yeah

... new thing, and your customers don't let you. So you are, you're-

Yeah, yeah

... you're, you're trapped.

Yeah, yeah. That, that, yeah, what you're describing is innovator's dilemma, right? Like-

... um, you know, this is the future, but your existing customers are, like, saying like, "I'm satisfied with my current thing." And the more you satisfy your existing customers, the higher the probability of you dying, which is like the irony of, uh, the innovator's dilemma.

Yeah. No, I mean, whether or not that actually applies to Arc and Dia, I don't know because-

Yeah

... it's just a feature. It's not a, it's not a product.

Yeah.

It's just a feature.

Yep, yep, yep.

But like, I don't know, you know.

I mean, I would literally change to Dia if they g- brought back the vertical tabs. And, uh, it's literally that one thing that's just, like, keeping me in Arc. Like, um-

Yeah, yeah. So, uh, the, the head of Eng is coming to speak at AIE and he's, he said, yeah, they're working on it. Yeah, they just, they-

Nice, nice. That's awesome

... they, they try to say no. They, they, you know, like, part of the product job is to say no even though people say they want it.

Mm-hmm.

They, you have to simplify, simplify, simplify, but I think the, the-

Yeah

... demand is too strong. Cool. Uh, well, thanks so much for, uh, you know, your time and, like, sharing your wisdom and, and stuff. Um, do you have any, like, close-up session, like philosophically or company-wise?

I'd, I'd say, um, I think there is, there is a lot of, there are a lot of questions around, "Hey, should I become a developer? Is the developer job gonna go away?" And things on those lines. And as much as it may look like it's gonna be self-serving for HackerRank or for me and things, I actually think you're gonna see a proliferation of developers.

I think you're gonna see way more developers in, in the future. I think the job role will change for sure. I think it's not gonna stay the same, but I think you're gonna see way more. In fact, we're actually starting to see developers are not...

Right now they're not even, like, contained into the typical R&D role. They're now starting to spread across the entire company. The hottest role right now other than AI researcher is probably forward-deployed engineers who are now, like, in, in, like, go-to-market.

And then the, probably the second hottest role, uh, or whatever, third hottest or whatever it is, like go-to-market engineer who are like, really like orchestrating all of these tools, whether it's Clay or, or, or any of them to be much more personalized outbound emails and stuff.

We, we do, we do quite a bit of that as well. So, uh, you're actually seeing the flip of it, which is engineering is actually entering across all of them, not just R&D. So I would actually say you-- we're gonna see proliferation of more Vibe coders, more production engineers, more developers across the go-to-market or other non-technical orgs, uh, or at least what are classified as non-technical orgs.

And I, I think it's, it's a shame that some of them are giving advice as to don't, don't learn to code and all of those things. I also think, like, coding is a way of sharpening your thinking. I think it's helped a lot even, like, when I think about, like, building a business.

It's always like an ambiguous problem. You try to make it much more concrete. You break it into sub-problems. You go ahead and solve that. That's a very effective skill for you to go ahead and, like, learn even in business.

So not only is it just gonna sharpen your thinking, just like writing just, like, helps you to make your thinking clear, um, I also think you're gonna see, like, way more developers in the future. So that would be my, uh, closing thought.

Go learn to code.

Yeah. Uh, I strongly endorse this. It, it's one of those weird things where, like, if you believe a little bit in AI co- in AI and coding, you do what you say. If you believe too much, then you're like, "Oh, it's gonna take away the job, so I don't bother."

And I think there's like a happy middle. It's, it's weird. I'm trying to get, I'm trying to get Singapore government to encourage people to, to learn to code. Like, I wanna change our national second language from Chinese to Python.

That is cool.

Yeah.

That is cool.

Like, ha ha, but like, no. Like, in schools, elementary schools replace every single like, like, at /S, /Chinese, /Python.

Uh-huh.

Right? Like, just search and replace and you will-

Yeah

... in, in 30 years you will have the strongest nation of coders you ever see.

Yeah. Yeah. Agree. Agree. Let me know if there's ways that you and I can partner on that. I'm working with, uh, the government of Rwanda, um-

What does Rwanda do?

... as a ministry who wants-- they wanna, like, create a million, uh, coders from Africa, um-

That, that sounds based

... um, from high schools and others. So-

So, like, but I mean-

And we-

... their, their practical, their practical thing is, like, they're gonna train all these coders, they're gonna spend all this time, and then where are the jobs?

Yeah.

Right? Like, so I think-

Yeah

... like, we have to, like, think about what you just said. I think it's the first time someone has explained to me that like, yeah, actually, basically every other function in the company is turning into engineering.

Yeah. Yeah.

So that's where the jobs come from, right?

Yeah.

It, it, it's just not the traditional sweet job. It is FDE that can code. And like-

Yeah

... I, I used to be in finance, and guess what? When I, when I was a trader, and I was young. I'm no longer young, but when I was young and all the older guys are like, "Hey, this thing requires some coding.

You look young. Can you figure it out?" And I did.

Mm-hmm.

And so I became a trader-

Yeah

... who coded, and then-

Yeah

... that became, it eventually got called a quant and, and then-

Yeah

... that's, that started my coding journey. But like I, I never had-

Yeah, yeah

... I don't have a software, I don't have a CS degree, nothing. Like I just-

Yeah, yeah

... you just learn it. Yeah.

Yeah. Skill over pedigree, right? That's, that's what matters.

Uh, cool. Well, it's a real pleasure, uh, and I'm sure we'll catch up, uh, in person and, and we'll-

Yeah

... we'll, we'll, uh, get you on the, the AIE stage at some point. But, uh, thanks for your time.

Yeah, yeah. Absolutely. Absolutely. Continue your great work. Thank you so much for the invite and, uh, yeah, looking forward to getting this live.