Oct 09, 2023
On October 5, The Capitol Forum held a conference call with IPVM’s Government Director Conor Healy to discuss IPVM’s reports that raise questions about the effectiveness of Evolv’s weapons detections system, including whether it is even capable of detecting knives and guns. The full transcript, which has been modified slightly for accuracy, can be found below.
KIM GEIGER: All right. Good morning. Thanks for joining our call. I’m Kim Geiger. I’m a Correspondent at The Capitol Forum. With me today is Conor Healy. He’s Government Director at IPVM.
Before we get started, I’d like to go over a few housekeeping items. You’ve joined the presentation, listening using your computer’s speaker system by default. If you prefer to join via telephone, just select telephone in the audio pane and the dial in information will be displayed. To submit questions to today’s presenters, type them into the questions pane of the control panel. We’ll collect questions throughout the call and address them during the Q&A session at the end of today’s conversation.
I’m going to first give some background about today’s call and then turn it over to Conor to introduce himself. The Capitol Forum has recently started reporting about Evolv Technology, a company that sells weapons detection systems to event venues, schools, hospitals and others that are trying to improve the safety of their facilities without disrupting the flow of people.
Evolv markets their products as walk through metal detectors equipped with AI technology that allows them to screen people as much as ten times faster, offering the premise of a seamless experience and selling that product at ten times more than the going rate for traditional metal detectors.
In the course of our reporting on this company, we found that IPVM has been looking into Evolv for years and has done quite a bit of really solid work looking at the technology. So we thought we’d take the opportunity to combine forces and host a discussion about what’s been going on with these detection systems, which are popping up all over the country and are often being bought with taxpayer dollars, despite plenty of media coverage of the limitations of the technology.
IPVM is a publication and research group focused on technologies in surveillance, physical cyber and supply chain security, AI security, analytics, facial recognition and more. IPVM’s Research Team operates a 12,000 square foot laboratory in Bethlehem, Pennsylvania for conducting, testing and analysis of these technologies. The publication is globally recognized as the leading source of security and surveillance technology research. It was founded in 2008 and does not accept advertising sponsorship or consulting projects from manufacturers.
Conor Healy leads IPVM’s investigations of global government use and regulation of surveillance, security, facial recognition, AI analytics and related technologies. He is an internationally recognized expert and peer reviewed author whose work has been cited by the U.S. State Department and Congress. Conor, thanks so much for joining us. I’m going to turn it over to you. If you could give us some background on how IPVM became interested in Evolv and a summary of your findings. Let’s start there.
CONOR HEALY: Of course. It’s great to be here. And thank you for the kind introduction. We’ve been covering Evolv for several years. I think since 2017. But we really started looking at them seriously in 2021 after they went public via SPAC. And the company came on the scene in a pretty dramatic way.
There are a few other companies in the weapons detection space, like CEIA, which is already a well‑established metal detector manufacturer, a company called Xtract One, which used to be Patriot One. But with Evolv, you had investors like Bill Gates, Peyton Manning, Joe Torre. They had marquee clients like the Atlanta Falcons, the Lincoln Center, the Metropolitan Museum of Art.
This was a pretty incredible marketing operation that Evolv was running. They were getting all these notable clients and investors. And so in the fall of 2021, we set up a call with some of Evolv’s executives to try to learn more about what they were doing and how the detectors worked. It was a bit of an unusual call for us in that Evolv was quite hesitant to answer our questions about how the system worked.
In addition to all the all the other incredible marketing, they were making incredible claims about what their detectors could do. But they weren’t really willing to tell us how they were managing to achieve that. And they wouldn’t let us test the systems. And to this day, that they still haven’t. Usually we buy something on the open market just to test. But in this case, you could only get the scanners through Evolv.
So we started looking at them much more closely, looking at these claims they were making. The CEO had gone on television and said in various other forums, like a TD Ameritrade interview, on Fox Business, that they have signatures for, quote, “every weapon in the world”. He said they had signatures for, quote, “all the bombs, all the guns, all the large tactical knives”. They were saying this is like a new class of product. It’s not a metal detector. It’s a weapons detector. And very importantly, that it could distinguish between rather innocuous items like cell phones and laptops versus guns and knives, which would be a tremendous innovation because it would make security screening really convenient if it did indeed work.
But what we were finding was that the claims didn’t quite stack up to what we were observing in the real world, what we ultimately heard from clients and customers, what we saw in Evolv’s testing with the National Center for Sports and Spectator Safety. These claims, as it turned out, we feel were misleading. And in some cases, that’s putting it mildly. And in fact, the ground truth was that Evolv could and did miss weapons. It could and did identify innocuous items as weapons. It had, and continues to have, problems with false alerts.
Over the past two years, we’ve tried to really kind of drill into what can Evolv do and what is it claiming it can do. There seems to be a really wide gulf there. And so, we’re at a place where it’s still being bought by schools. And I think there’s still a lot of good reporting yet to be done on the company.
KIM GEIGER: Your organization recently called on the FTC to look into Evolv. Can you kind of walk us through what specifically you’re asking them to look at and what your rationale is for that?
CONOR HEALY: Yeah, sure. So I think if you were to sort of boil down FTC rules on advertising and marketing to a single statement, it’s don’t deceive people. Don’t lie, basically. Based on the evidence that we have, Evolv’s marketing claims may have run afoul of those rules, perhaps quite significantly.
So to give a current example, if you go on Evolv’s website right now, on the homepage, there is a claim that it’s scanners and I quote, “eliminate the friction” unquote, of screening for people such as students. But in fact, as your own report showed, and as multiple other accounts at schools show, there is a serious problem with false alert rates. There’s actually quite a lot of friction. We’ve seen schools that have 30 to 40 percent false alert rates where those students have to be pulled aside and go through secondary screening, have their bags searched. Or we see schools that use Chromebooks, which are extremely common in schools, as laptops. And they seem to regularly get flagged as weapons by the detectors. It has to do with the hinge in the laptops.
And so, our concern is if Evolv is saying they eliminate friction, but there’s all this friction, is that a fair claim to be making? And that exists in a constellation of a number of other claims. The CEO’s claims about having signatures for every weapon in the world, and that in testing not really being borne out. We’re talking about tens of millions of dollars in taxpayer money. We’re talking about significant expense also by private businesses, by Disney World, by a whole bunch of different stadiums. So there is harm that potentially is being done here. And that’s why we think it’s something the FTC should at least look at.
KIM GEIGER: I know that one of the things that the company is kind of relying on is this report that was done by the National Center for Spectator Sport Safety and Security. So they kind of point to that as this independent third party testing that has proved their system to be effective. If you can kind of just walk us through, I know you guys have kind of poked holes in the their characterization of the report. But I also just want to get some background on like what is this organization?
One of the things that Evolv told us was that, yes, they had paid for this study and reports have been done, but that anybody who wants their product studied by this organization has to pay for it. And then another thing that you guys have mentioned in a lot of your reporting is that Evolv actually designed the structure around the testing. So they kind of set it up to be able to make their product look like it’s performing as well as possible. And is that sort of like an industry standard? I know you guys look at technology like this all the time. Have you seen this happen with other technologies that have been tested by this particular entity?
CONOR HEALY: Yeah. So, I mean, the fact is that the report produced by NCS4 on Evolv’s technology did not disclose that Evolv had paid for that testing. It also didn’t disclose that Evolv had paid for $10,000 luncheons for NCS4. Evolv’s excuse seems to be, okay well, everyone does it. And so maybe these can be looked at as two separate issues.
Should NCS4 be paid for its testing and not prominently disclose that in reports? And that might be an NCS4 issue. But to tell you a bit about the organization, it’s the National Center for Sports and Spectator Safety. It’s based out of the University of Southern Mississippi, which is a public university. So it’s a kind of quasi‑government agency, NCS4. And their goal is to provide information on security best practices, in particular for sports and spectator safety in stadiums and other such venues.
And part of what they do is, as they did for Evolv, they do this testing of security technologies to see how well it works. And they do it through what they call an operational exercise. Basically, this means that they take the technology and they deploy it in the real world and they see how well it works. So in Evolv’s case, Evolv provided detectors and they set them up at the entrance to a soccer match in Columbus, Ohio, with the Columbus crew soccer team. And that’s where they then went and collected their results.
Aside from the issue of who paid for the testing, when this report initially came out, it was heavily redacted. It was a 20-something-page report and made it clear in the disclosures that quite a lot of material had been removed from the report, that there was a full version which was being kept confidential. And they claimed that this was for security reasons. Because if they provided the full report, it might let bad actors have a guide to how to defeat the system.
We hadn’t seen that before in the NCS4’s reports. NCS4 is pretty well known in the industry. My sense is that they have a fairly good reputation in the industry. And so what they say does carry some weight. But this was the first time with these operational exercises we’d seen them withhold information from the public. And so, of course, we became very curious.
And because NCS4 is part of the University of Southern Mississippi, they have to respond to Freedom of Information requests. We made one for all the correspondence between Evolv and NCS4, as well as the full report. The latter was denied. But when the correspondence was sent to us, they also accidentally sent us a whole bunch of documents that they had intended to withhold. And it turned out that material they had deleted from the report was quite alarming because it directly contradicted Evolv’s marketing. It turned out that Evolv had gone through 14 separate rounds of drafts with NCS4 where their executives were red lining things and removing items from the report.
The testing showed that Evolv did not reliably detect knives. For some types of knives, it had a zero percent detection rate. Overall, it was, I believe, a 53 percent detection rate. This was deleted from the report.
In fairness to Evolv, the CEO said they had signatures for all the quite “large tactical knives”. So maybe we can give them some leeway there.
But the report also showed—and again this was deleted from one of the evaluators comments—that the system only detects ferrous metals, which is a lot more interesting than it sounds. That means that basically it detects what a metal detector would detect. It detects metals. And Evolv’s CEO had gone on Fox Business and gone on TD Ameritrade, he sat on TD Ameritrade, and I quote, “We’ve written the signatures for all the threats that are out there, for all the guns that exist. So all the guns, all the bombs.” Well, a lot of bombs don’t necessarily contain ferrous components. Plastic explosives wouldn’t be detected. Pipe bombs that are made with lead wouldn’t be detected. So this clearly showed that that was a lie. There were a number of other issues. Sorry, go ahead.
KIM GEIGER: Can you sort of explain that term terminology to me? It’s something I’ve had a hard time grasping, written the signatures for weapons. What does that mean, written the signatures?
CONOR HEALY: So that’s actually a specific scientific term. And if we had my colleague Nakita on the phone—who I work together with on these investigations and he’s a master’s in physics from Yale—he would probably give you a much more detailed explanation. But in short, my understanding is that that is like a specific profile of a given object that is taken as it passes through a magnetic field.
So it refers to the specific ability to detect that item. That’s at least sort of it in scientific speak. Is it meant as a scientific term? We’re not 100 percent sure. But I think it’s quite clear what the implication is of his comments, my interpretation is he’s saying we can detect all these items. And even if that’s not what he’s saying, just the claim itself, to say we have the signatures for all the weapons in the world, seems very unlikely to be true. I mean, do they have signatures for every possible variation of a 3D printed gun, for example? Maybe that’s a rare case. I’m not sure school students are going to bring 3D printed guns. But look, if you’re saying that you have signatures for every weapon in the world, I think you better be able to back that up.
KIM GEIGER: So I am wondering about like I know that they say that they can detect, you know, they can move people through ten times faster and they have this special AI technology. Did you guys find that their systems are any more effective than like a traditional standard metal detector that we see all over the place?
CONOR HEALY: Well, I’ll preface by saying, again, Evolv hasn’t let us test their systems. I think that given that limitation, we’ve done a great job of assessing how well Evolv works, but we haven’t been able to do a side‑by‑side comparison. I think a better way to look at it is that there are sort of benefits and drawbacks to a system like Evolv versus metal detectors.
So when it comes to the ten times faster claim, that we’re quite skeptical of. The throughput of a metal detector or of Evolv’s weapons detectors depends on a lot of factors that have nothing to do with the detector. It depends on staffing levels, for example. It depends on the level of screening that’s occurring. Like the throughput at an airport security checkpoint is going to be different than the throughput at a baseball game. And that is not wholly because of the type of detector that they’re using.
And with the ten times faster claim, Evolv will not tell us what that means and where it comes from. It would be, I think, reasonable to expect an asterisk saying that this is based on sort of X and Y scenario. In reality, we don’t think it’s like a reasonable claim to make.
In any case, with weapons detectors, you could describe them as like a somewhat smarter version of metal detectors. But really what they are is a trade off, right? With a system like Evolv, you are going to detect many weapons. I want to be clear about that. We’re not saying that it’s vaporware. And compared to a metal detector, you may have fewer false alerts on things like phones or laptops. And it would be fine if Evolv marketed it in that measured way. The problem here is that they’re creating a false sense of security.
Instead of saying, look, this has advantages potentially in some situations versus metal detectors. You know, not in airports, but maybe in an environment where security is not quite as heightened, potentially a school. That’s sort of up to the individual school, maybe it makes sense. We’ll have fewer alerts on things like cell phones and we might have fewer alerts on weapons too. But it could overall be a more reasonable, efficient process.
Instead of all that, they said, you know, we have signatures for all the weapons in the world and we eliminate the friction. You know, we can tell the difference between a cell phone and a gun or a Chromebook and a bomb. It’s that kind of categorical, aspirational, utopian marketing that is fooling people.
KIM GEIGER: Yeah, on that topic, I want to talk about those limitations involving the Chromebooks, the water bottles. It seems like customers are buying these systems under the impression that they’re going to allow people to just go through. And then they’re finding after the fact, like after they’ve bought and installed the systems, that making that happen is going to require them either to have people pass things like Chromebooks around the systems so that they don’t alert or lower the device sensitivity to the point where it might not be able to detect an actual weapon.
If you could sort of walk us through like how long have these issues been known? We recently reported about a school district in New Mexico that encountered this problem just at the start of the school year. But I know IPVM has been reporting about the Chromebook thing for a while now. And it seems like the company has known this for at least a year or two years, that this is an issue.
One of the things that we found actually with the New Mexico School District was that when this issue came up, the company was right there with a brochure about how to install like a conveyor belt system on the side of the detector that people could then put their laptops and binders on as they walk through. So it’s not like it was a surprise to them. And I’d like you to kind of just walk us through what’s been known for how long and what exactly are those issues?
CONOR HEALY: Yeah, in that situation where there’s a brochure for a conveyor belt system, I think it is quite duplicitous, right? It’s like you’re saying, okay, the system will do X and Y, then you buy it and it turns out that you also need to get Z product right? That seems not totally fair to the buyer, particularly when you’re spending ten times what you would be spending on metal detectors.
Look, I can’t sort of enter inside the mind of the person who’s buying this. It may be the case that in many of these school districts, they went along with the marketing in public. Because there are often public releases that kind of tout the same claims that Evolv makes. But privately, maybe they knew that this was exactly what was going to happen.
I find that difficult to believe personally. Your report showed that it’s not just Evolv’s marketing in public where they say that they eliminate the friction of screening, that they go ten times faster than a conventional metal detector. They’re also saying this directly to the schools that they’re marketing to.
Now, Evolv says, and has told us, that they’re fully transparent with their customers. But all the evidence we’ve seen doesn’t show that. We’ve also, separate from your report, looked at sole source documents that have touted the same misleading claims. And so even if we give them the benefit of the doubt that school security managers are at some point in the sales process told the truth about how these detectors are actually going to work and what’s actually going to happen, surely this has an effect on what school administrators decide in terms of these purchasers, what school boards decide, and also what the public thinks about the expenditure of millions of dollars of taxpayer money.
So, although I’m not convinced that they are actually transparent about these limitations, I think even if they are, we still have a big problem here. Because I don’t think it’s fair to say one thing to the public, to say this is going to work in X and Y way, but then privatelysell something different to a government agency using taxpayer dollars. And that’s just not how the kind of public consultative process of procurement is supposed to work. And I don’t think it changes the fact that these are still false or misleading marketing claims.
KIM GEIGER: It seems like when someone like a school district buys this product and then finds that, okay, it’s actually we’re getting all of these alerts, this is actually slowing things down. And then the recommendation, at least from what we’ve seen, is, okay, pass these things around or lower the device sensitivity. Sometimes the recommendation for how low to go on the device sensitivity enters into the realm of what I think was shown by that NCS4 group as no longer being capable of detecting things like knives. So it seems like there’s a tradeoff here once you’ve installed the system where either you are going to be dealing with all of these false alerts or you’re going to be just passing people through the system and not able to detect anything.
CONOR HEALY: Yeah, exactly. And by the way, a very similar tradeoff exists with conventional metal detectors. And it’s one of the many moments where you stop and think like, okay, so what are the differences between this product and a metal detector? Yeah, it’s absolutely a trade off. If I’m not mistaken, the NCS4 testing was done at setting E. And that, at the time, was the second highest setting, highest being F, and it still missed many knives. As you lower the sensitivity setting, one can expect you will miss more.
KIM GEIGER: And on the topic of missed weapons, there have been incidences where people have taken weapons through the Evolv system. Can you walk us through those instances?
CONOR HEALY: Yeah, sure. I mean, the most prominent one occurred in Utica, New York, at Utica schools. They spent $4 million on Evolv detectors. And a student was able to bring a rather large knife through the system and then stabbed another student tragically and quite seriously injured that student. The school district at that point decided to remove the Evolv detectors. And that $4 million expenditure was not refunded to them. Last I checked, the school board was trying to sort of sell their lease, as it were, or sell their subscription, to a nearby hospital in order to recoup some of the costs. So, even when a student gets stabbed, just as a side note, you can’t get out of the subscription that you’re paying to Evolv.
KIM GEIGER: Can you talk about the subscription model that they have?
CONOR HEALY: Sure.
KIM GEIGER: It’s not just a one time payment, like you buy this thing and then you use it. You’re actually paying on a contract.
CONOR HEALY: Yeah, of course. So with Evolv, you don’t really buy the system. They’ve changed the model a bit over the past couple of years. If I’m not mistaken, the current approach is that you pay for four years of use of the system. And the idea is that they’re selling this as a SAAS product, sort of as a software subscription basically. Whereas, conventional metal detectors are products that you purchase. Sort of as simple as that. And they cost about ten times less than that subscription.
KIM GEIGER: Do you have any sense why, like in the case of the Utica stabbing, there haven’t been any lawsuits?
CONOR HEALY: Well, I’ll say there haven’t been any lawsuits yet. I’m not personally familiar with what that student’s family might be thinking, but I wouldn’t be surprised if sometime down the line they did take some kind of action. As far as any suits between, you know, by the school district and Evolv, my sense is that they have a contract that they probably feel that they can’t get around. But I can only speculate.
And what’s important about Utica is it’s one of the very few cases where there was any public acknowledgment by the school district that they may have—or that they did, in fact, make an error in purchasing Evolv. So even if there’s no lawsuit, it’s still actually remarkable. The school superintendent said his opinion was that Evolv is not suitable for schools. And the reason that happened, it would seem, is that the Evolv purchase was made by a prior superintendent who, for reasons that are unknown, had to leave his post and was replaced by the current superintendent. So, the guy in charge is not responsible for the purchase and I think was probably freer to say what was really on his mind.
KIM GEIGER: Yeah, internal school politics.
CONOR HEALY: Yeah.
KIM GEIGER: I have a question here from a caller. Can you talk a bit about the work arounds where people take stuff out of their backpacks and place them above the Evolv sensors, effectively preventing large items from being scanned? Is that a good practice? And can’t someone just hide something in a binder?
CONOR HEALY: Yeah. So, first of all, these workarounds demonstrate that there are actually some issues to be solved with the Evolv, contrary to the marketing. The idea is that if you hold something above your head, basically, either you hold it above the Evolv sensors or you hold it in a region of the sensor where the sensitivity, depending on what the school’s approach is, where the sensitivity’s been reduced and it’s less likely to create a false alert.
One challenge of this with schools is not all school students are going to be tall enough to achieve this. And it still is a far cry away from, you know, what Evolv said to the public and to investors, which was that you don’t have to remove items from your backpack. But it could be a solution that to an extent reduces false alerts.
KIM GEIGER: Another question from a caller. You raise these marketing issues for Motorola, which is a backer of this company. What happened or what changes did Motorola make to its marketing claims regarding Evolv after you raised the marketing concerns?
CONOR HEALY: So Motorola has a page about Evolv on Motorola’s website. And on that page, it made that claim that it works ten times faster than a conventional metal detector. We reached out to Motorola to ask them about that claim, and it was deleted from their website. We didn’t get any response from Motorola, but that occurred shortly after we emailed them. So, because they didn’t provide any comments, I can’t speak to precisely why they did that. But this is not uncommon with the companies we report on. We very frequently will find issues where marketing does not quite make sense or is misleading. We do reporting on a whole host of different companies. We try to be objective and independent. And very often, when asked about those issues, not always, but often they might make a correction. Evolv is not one of those companies. They pretty much stuck to their guns on most things.
KIM GEIGER: But they have made some shifts right where they went from saying like weapons free to safe zones to safer zones?
CONOR HEALY: Yeah.
KIM GEIGER: Have those changes correlated with your reporting or with any incidents that have happened?
CONOR HEALY: One senses that they have correlated with our reporting. Evolv didn’t specifically tell us that that was the case. But we were pretty focused. Like their banner marketing was weapons free zones. That Evolv creates weapons free zones. If you Googled Evolv, immediately you would see Evolv weapons free zones. And that was kind of a version of what the CEO had been saying publicly.
And we were kind of like, look, you know, we have these test results. We have all these reasons to think that, well, you don’t create weapons free zones. You catch some weapons. You catch many weapons. But they’re not weapons free zones. And then they changed it to safer zones. And then later changed it to—sorry, they changed it to safe zones and then later changed it to safer zones.
The only explanation Evolv really provided was we believe that safer zones sort of—I’m going to butcher this—but kind of captures Evolv’s mission. But they didn’t explain actually why they made the change.
KIM GEIGER: Another question from a caller is about health concerns involving this device. There was a report about a woman who had an implantable cardiac device, I believe.
CONOR HEALY: Yeah.
KIM GEIGER: Who had gone through an Evolv system and said that the device then stopped working. Do we have any indication—I know the company says that they are they don’t interfere with devices like that. But is there any indication that that’s not true? Or that people with something implanted in them should be careful going through those things?
CONOR HEALY: Well, I’ll say that people that have something implanted in them, I think they should be careful going through Evolv scanners or any scanners, just as a general matter, irrespective of what the company says. I would probably ask my doctor. Personally, that’s what I would do.
The issue is a bit confused because on the one hand, my understanding is Evolv said that it doesn’t interfere with these devices. But Evolv did also say that you should consult your doctor to make sure that you can go through systems like this.
I can’t speak to the specific claim where this individual said that her device stopped working. What I can say is that it is indicative of a situation that could occur because of this marketing that Evolv has that it’s a weapons detection and that it’s not a metal detector.
The claim is that this woman didn’t want to go through the system because she couldn’t go through metal detectors and then was told that she had to and was told that it wasn’t a metal detector, which is what the marketing says and what the staff at that stadium probably understood to be true. One could imagine the situation where someone thinks it’s not a metal detector and maybe they have something that can’t go through a metal detector and that ends up causing an adverse effect. Like I said, I wasn’t there to witness it, and I can’t back up her story. But this is something you have to think about when you market products like this.
If I could just add one thing. These are security products, you know. This isn’t Candy Crush. This isn’t an app or some Silicon Valley invention. I think it’s important to keep in mind that every time Evolv sells a system, I don’t know about their legal liability or their contracts, but they are taking on the responsibility to protect people with their technology. And that’s something I think that they should take very seriously.
KIM GEIGER: Speaking of legal liability, they have this special designation from, I think, the Department of Homeland Security.
CONOR HEALY: Yeah, the Safety Act.
KIM GEIGER: Can you kind of explain what that is and why companies with these types of technologies bother to get that? And kind of like the dual purpose of it from a liability, but also from a, I guess, marketing standpoint.
CONOR HEALY: Yeah, sure. Dual purpose is a good way to put it. So strictly speaking, when you get a DHS Safety Act designation, what that means is that DHS has determined that your technology has some kind of like anti‑terrorism application. And as a result, it makes you immune or less exposed to liability from a mass casualty event in which your system sort of didn’t prevent it. And so Evolv is not really used at airports. But, for example, let’s say at a stadium if someone got through with a bomb and there were mass casualties events from that, their DHS Safety Act designation, one assumes, would indemnify them from any claims related to that event.
But like you say, it has a dual purpose. What it sort of really is when people hear about DHS Safety Act designations is it’s a marketing tool. It’s doesn’t actually represent approval from DHS. DHS is not saying we love this product. You should buy it. But they do provide like a DHS Safety Act approved kind of logo. And companies will often sort of market this and people read it and kind of assume that the designation is some form of approval. And that, I think, is a big reason that many companies seek to get this.
KIM GEIGER: Interesting. And not to sort of beat this horse, but what is our sense of whether or not Evolv is capable of actually detecting weapons that could like bombs or things that could be used in mass shootings, things like that? Does having an Evolv system even theoretically make a place safer than just a standard metal detection system or guards with metal detecting wands sort of thing? Or do we know?
CONOR HEALY: Okay. So, I mean, I think it makes a place safer in the absence of other things. Our sense is that it doesn’t make a place safer versus a metal detector or sort of more conventional screening methods. Evolv would likely detect weapons that come through several different specific types. If you made a pipe bomb out of something ferrous, like iron let’s say, rather than led, that’s the kind of thing that would be picked up by a metal detector. And I haven’t seen testing on this, but I think there’s a good chance it could be picked up by Evolv too. The same with guns that have ferrous components.
So, yeah, there is absolutely a possibility that Evolv can pick things up like this. But like a metal detector, it has that same limitation where it can’t pick up plastic explosives. Which in fairness, are not, as far as I’m aware, super commonly available to the public. But lead pipe bombs as well. I mean, these are also not hypotheticals. Evolv is used at the same Manchester Arena where at an Ariana Grande concert a few years ago bombs were set off as part of a terrorist attack.
KIM GEIGER: I want to ask about the AI technology claim. This is one that they say we are equipped with this special AI technology, which I think kind of goes along with the like written the signatures of all the weapons in the world lingo does. What does this mean, AI technology? Do you have any sense of what exactly that technology is? Is it even a real thing? Does it work?
CONOR HEALY: Yeah. So we really only have the biggest sense, I would say, of what Evolv is using AI for in this detection. And what seems to be happening here is that Evolv has done some like machine learning and training where they’ve taken various weapons and other innocuous items through the system and sort of told that system like this is X weapon and this is actually sort of Y innocuous item. And if you do that enough times through machine learning, you might be able to make the system better at determining what something is.
Now, Evolv is pretty vague about how AI factors into its systems. They also say that it’s constantly learning. That’s not really true on the level of the user. You know, the user gets software updates, right? The system doesn’t change any of its functionality on any ongoing basis. So, I mean, Evolv implies that like it’s constantly learning in the real world, but that’s not quite true.
What’s actually happening is Evolv says they’re collecting data from its deployments with the permission of the user. One could expect they don’t get much useful data out there in the real world. I mean, for it to be useful, it has to be properly collected and categorized. If a gun goes through, like you have to tell the system that was a gun. Maybe you have to tell the system what type of gun it was. If a phone goes through, is that false alert? If it works, is it properly tagged?
I question whether, for example, school security officers or teachers can really be relied upon to tag data really thoroughly and properly when you have sort of throngs of students trying to get to class. But that’s a sort of an issue for Evolv to resolve, whether or not they can improve their AI, as they call it.
Another issue just at the moment is like if it’s artificial intelligence, does it have to be intelligent is I think an interesting question here. Because the on the ground data that we have indicates that Evolv often does not distinguish between weapons and innocuous items. So if it is artificial intelligence, it seems like it could be a lot more intelligent.
KIM GEIGER: Okay. This is the last question I have from callers. If anybody has any more questions out there, please submit them. This one is do you have a sense of whether the FTC is interested in this issue or is likely to do anything here?
CONOR HEALY: I don’t know whether the FTC is specifically looking at Evolv. And that’s not the kind of thing that they would disclose publicly. What I can say is that the FTC has been, I think, very interested in claims surrounding AI, and that they have also been very interested in the security industry, particularly in biometrics technology, which doesn’t specifically apply to Evolv. But I think the FTC is pretty interested in the security technology area. Whether or not they will do anything with Evolv and what that might look like is a question really only they can answer.
One argument that Evolv might make is that their marketing is like business‑to‑business rather than business‑to‑consumer. And also, that many of their customers are government agencies, which—and I’m not an attorney, but my understanding is that the FTC typically does not focus on false claims made to government agencies.
That all being said, I don’t think that will carry a ton of weight, either of those issues, with the FTC. There are many cases out there that have upheld the FTC’s ability to go after issues in the sort of business‑to‑business space and that businesses can still be classified as consumers. But I’m just mentioning that as an argument that could come up from Evolv’s side.
KIM GEIGER: Is there anything we haven’t touched on that we should be thinking about here?
CONOR HEALY: No, I mean, I think that about covers it. I guess the only question is why do schools buy Evolv, sort of given the reporting that’s out there? And I think that that’s sort of an interesting thing to examine just for the security industry at large, which often makes money when people are scared. I don’t mean that to malign Evolv in particular. I think that in the industry in general, security is about protecting people. And if people feel unsafe, you sell more security products. This is sort of unsurprising.
I think schools are very scared. Students are scared. Parents are scared. And that creates an environment where it is sort of much easier than it used to be to spend millions of dollars on something that people hope will protect them.
KIM GEIGER: That’s definitely, I mean, I talked to one school board member who said that that was his experience, that he felt like his fellow board members, once they once the idea was there, we need to have some way to make the school safer, then not doing that wasn’t really an option. They needed to go along with it. I think he was the only person who voted against it.
CONOR HEALY: Yeah, and that’s sort of what we’ve seen in a lot of school districts. And they do tend to defend their purchases, even if they end up sort of not—like in the Utica schools—not really being what they expected. As I was saying, Utica was an unusual case because of that change in the superintendent.
It’s a –I don’t know if I’m using this term correctly, but it’s a double edged sword for Evolv, so to speak. The fact that they can sell the school makes them a lot of money. But selling to schools means that when there are problems with the products, they tend to get exposed. They tend to come up. People report on incidents that happen at schools. Students take videos and put them on Tik Tok and so on. This is not really an environment where it’s very easy to cover up issues with the systems. Not to mention public records request.
KIM GEIGER: Yes, true. All right. Well, that’s all the questions that I had. Conor, I really appreciate you giving us so much time and sharing all of your expertise on this topic. You guys have been doing some really great work and looking forward to seeing what else you do on this.
CONOR HEALY: Yeah, of course. And my kudos on your report as well, which was excellent. So it’s been a pleasure speaking to you. And thanks for having me.
KIM GEIGER: You too.
CONOR HEALY: All right. Thanks again.
