2024 Air, Space & Cyber: Hacking the Acquisition Bureaucracy

“Hacking the Acquisition Bureaucracy” featured Rupak Doshi, CEO and co-founder of OmniSync; Robert Fehlen, chief strategy officer at Mobilize VISION; and Bryon Kroger, founder and CEO of Rise8. The discussion was moderated by Alexis Bonnell, CIO and director of the Digital Capabilities Directorate at AFRL, during AFA’s Air, Space & Cyber Conference on Sept. 16, 2024. Watch the video below:

Panel Moderator: Alexis Bonnell, CIO & Director, Digital Capabilities Directorate, AFRL:

Welcome to Hacking Acquisition. I think that’s one of everyone’s favorite dream topics and we get to talk about it today.

For those of you I haven’t met, my name is Alexis Bonnell and I’m lucky enough to have the honor of serving as the Chief Information Officer and the head of digital capabilities for the Air Force Research Lab, where we get to serve both airmen and guardians, bringing amazing new technology to bear.

But as we know, bringing technology to bear can be really hard when you are up against some of the processes, protocols, and other types of things. So today we’re gonna have a candid conversation, even a little bit spicy if we can, gentlemen, on how do we do that better? And, you know, so I’m gonna go ahead and ask, we’re gonna jump right in. This is gonna be very flowing, and we’ll, we’ll get into it. So, you know, if I can ask each of you just introduce yourselves and why hacking acquisition matters.

Dr. Rupa Doshi, CEO & Co-Founder, OmniSync:

Hi everyone. My name is Dr. Rupa Gochi. I’m the co-founder and CEO of Omnis Syn. We’re from San Diego and we essentially have a mission to help ideas get to markets faster. And we do that on two ends. On the one end, we help companies and ideators find opportunities within government and industry. And on the other end, we help industry and governments find the right and the best technologies to access regardless of where they come from.

Robert Fehlen, Chief Strategy Officer, Mobilize VISION:

Robert Fehlen, chief Strategy Officer for Mobilized Vision. Very excited to be here. First off, thank you. My director of PR said the best time to schedule an acquisition hacking conversation is right after lunch, ’cause that’s when people are most energized, not ready for naps at all. So for your sake, I hope this is great, and for his job’s sake, I hope it’s great as well. My background’s private equity. Venture capital was a head analyst on two and a half billion outta the Midwest. Built and sold two companies in the industry space. To answer the question on why I believe hacking acquisitions is important, my grandfather, who just recently passed away, always reminded me that I was the first first-born in our family not to serve in the military directly. And it wasn’t for lack of trying due to health considerations. So me getting to come back to this industry on the commercial side and get to serve and participate in this way just creates a deeper passion. I see the importance of acquisition, getting the technology and the capabilities to the warfighter as soon as possible.

Bryon Kroger, Founder & CEO of Rise8:

I’m Brian Kroger, founder and CEO of Rise Eight. We continuously deliver valuable software that warfighters love. Why acquisitions matters to me is I was actually an intelligence officer in the Air Force for 10 years. I did mostly targeting operations for the first seven years. I like to say that I used really terrible software. Some of you might be familiar. After a particularly egregious targeting incident, I went and looked back at what I thought was an issue with my targeting software and traced it all the way back to requirements that had been generated eight years prior. The development had actually completed a year prior to this particular incident, and the software was sitting on the shelf waiting to go through the ATO process. And so, I felt like the acquisition process in this case actually got people killed. That’s what led me to leave targeting and go to acquisitions, where I was one of the co-founders of Kessel Run. I’ve been on a journey ever since to continue hacking the acquisition bureaucracy.

Alexis Bonnell:

So, I mean, let’s talk about that. Let’s jump right in after that. One of the things that we are facing is, in essence, we’re living in an exponential age, right? This is not an age where we have, you know, a five to 15-year change horizon. I mean, as you highlighted, we’re seeing, you know, Russia counter Ukrainian software in two weeks. So, we don’t necessarily have the same game board as we play that. So, for us, you know, we’re looking at that and saying, okay, one of the things that really introduces is in some ways, it introduces this concept that time is actually the ultimate weapons platform, right? And if we’re going to harness that, what are some of the ways that we get that time back? You know, I was talking to someone, and they said, you know, what is acquisition? My best thing, the way I could describe it was, you know, acquisition is where imagination and action meet, right? And one of the things about right now is we’re just imagining faster, right? We’re adapting faster. So, my question to you all would be, what does that look like then from a government lens, when we think about hacking acquisition, what are some of those hacks that we have to be thinking about if we truly accept that time, in fact, is a weapons platform, and that we’ve got to have a different relationship with it?

Dr. Rupa Doshi:

Yeah, I think one thing that everyone has to realize here is that in today’s day and age, acquisition really has to happen not based on convenience, but for competence. It’s extremely important. And when that happens, you basically are in this process where you have to cut corners where you can and where it’s possible and where the government gives you leeway to do that. And as much as everybody points to the FAR as this really difficult system to navigate with all these rules and regulations, actually what we found out is that the FAR is actually quite loose and flexible in terms of a lot of things, in terms of the ways in which terms are interpreted. When it comes to market research or when it comes to RFIs that people put out, you can basically turn an RFI into an RFP without too much trouble. And so those are the kinds of hacks that I think contracting officers and requirements officers, resource advisors, and folks have to be open and cognizant to when working with companies and trying to access the latest and greatest that’s out there. And a lot of times, the companies themselves can teach you a lot of these things because they are the ones who’ve done this time and again with multiple other agency partners as well. So be mindful of that discussion and that very candid conversation with these companies when you’re talking to them about that. Like, what’s the easiest way to get that technology?

Robert Fehlen:

Yeah, I’ll just add to that. In our team, we always talk about, hey, our customer, we need to train our customer as the experts in their problem. Like, the person closest to the problem is typically going to be the expert in said problem. But for us, on the outside, we need to be the expert in the solution in many cases. And it’s a collaborative effort. So some of the challenges on the industry side—so if I’m, you know, recommending a hack on the government side, it’s to start to look to industry as the experts in this potential solution or the capability versus, you know, what we face a lot of times right now is a list of requirements: “This is exactly how, you know, this piece of software needs to work,” down to a very granular detail. And there may be ways that we can, on our side, cut corners, get to a solution faster that can scale better if we were able to operate more in a collaborative environment versus having “Here’s the list of requirements, here’s how you got to deliver it, here’s the costs associated with that.”

Bryon Kroger:

Can I do like five hacks?

Alexis Bonnell:

Get in there?

Bryon Kroger:

I’ll bring up another incident that I just recently saw when I was out in theater. There was a counter-UAS system that didn’t meet the need, and we had a UAS strike on US soldiers. One thing that really struck me about that is the same problems that I saw back when I was a targeter, and everybody was looking at the system and they’re saying, “Hey, but there’s this program of record.” You know, there’s—maybe I shouldn’t name names—but there’s a technology that I think could meet the need absolutely 100% today that Anduril has produced. Unfortunately, there’s a big program of record in place and everybody says, “Well, we can’t procure that system because we’ve got this program of record over here.” Programs of record, first of all, is like a very loose term. We should be careful how we use it. Program dollars are tied to requirements, not to contracts. So, if you have a failing contract, that is not a program of record—that is a contract, and you should cancel it. Hack number one: cancel failing contracts.

Alexis Bonnell:

Cool. And even if it’s not failing, it just may not be what the moment needs, right? In this ever-changing environment, it may not be failing, it may just be like the demand shifted, right? The adversary shifted. Sorry, go ahead.

Bryon Kroger:

Yeah. And then maybe one other that I’ll mention is making, you know, as you’re trying to do innovation efforts, trying to make your thing look like the old thing. So one hack that I used way back in the early Kessel Run days before we had section 804 and 806 and the software acquisition pathway was we drew what we were doing as one of those acquisition pathways straight from the DOD 5000 with the slanted lines and the milestones, and we decided, you know, we used a VC growth board method for what would be like a normal milestone. So, we replaced milestone A with one kind of growth board, milestone B with another, but we made it look exactly like the old thing. A lot of times communicating in the language of the existing system can really help you get through the process.

I would just say you can’t hack a system you don’t understand. To your point, you know, knowing the FAR—Dan Ward famously said, “The ignorance of the FAR is a greater barrier to innovation than the FAR itself.” Highly recommend checking out his FAR guide, Dan Ward’s FAR guide. You can Google it and you’ll find it. Tons of hacks in there for hacking the FAR.

Dr. Rupa Doshi:

Can I add to your point, Alexis, real quick?

Alexis Bonnell:

Absolutely.

Dr. Rupa Doshi:

So, you mentioned that, you know, the contract or the technology may not be the right one for the time anymore, be it, it was two years ago, a year ago. DLA actually has a very, very interesting program right now that they kicked off that, that I read the other day, that they were actually conducting a whole planned obsolescence project for new technologies right from the get-go in their supply chain. Every single element and item and part number, they’re working on a way to predict the time when that technology or piece of hardware could become obsolete. And they’re already putting plans in place to do the research ahead of time to get ready.

Alexis Bonnell:

That really resonates with me so much. One of the things that’s so interesting, of course, when you get to be at the Air Force Research Lab, right? We get to see so many amazing things, and part of our job is actually to understand what do we not need to create, right? Because it’s already out there, and I can’t tell you how much I love getting calls, you know, from Global Strike or PACAF or whoever’s saying, we think there’s this use case for AI and being able to say, you’re right, and it already exists, and you should go get it. So, right, sometimes that idea of awareness alternatives, but to your point, we’ve done some practical exercises in trying to make sure we’re really thinking about these things.

And what I find to be interesting in, in any government organization, but especially in DOD, we often think about, and we really value started doing things right, and we maintain doing things really well. How often do we stop doing things right? And if you think about it from an innovation, you know, from having that resource, you have to stop, right, in order to start.

And one of the practical things that my team will do when we look at a new technology or a particular thing we might go after, is we actually, the first thing we do is we write a press release, right? Not one that we actually plan on sending out, but what the exercise does, to your point, Bryon on story, is it lets us think through, you know, in six months, in a year, in two weeks, whenever, what does success look like, right? For all the stakeholders, what would we want to say?

But to your point, we also write a pre-mortem. We actually ask ourselves, when and how would we know that this is supposed to end? Is it because it was overtaken right, by a better technology? Was it because it didn’t work the way we expected it to? Was it because, you know, any other number of things, but we find that there’s a real kind of gift we give ourselves, right? In our roles and our leadership in our humanity, when we’re actually able to say, it’s okay, we know it’s not going to be relevant at some point or forever.

So, you know, I wonder from you all just really quickly, what are some of the traits when you think about someone who’s thinking about requirements or a really great, you know, if you could wish characteristics or behaviors on the future state of great acquisitions people, what would some of those characteristics be?

Dr. Rupa Doshi:

I think the first thing I would always want to see in an acquisitions person is, uh, objectivity. That’s, I always like, have full faith in the process in trusting the process where if you go in with an open mind, you’ll probably find the best that’s out there, as opposed to going in with preconceived notions for something that you think is how something should work. Or maybe you’ve asked someone and that’s your view of how things work.

So, objectivity is the first, and second one would be an analyst mentality to really understand the data that is coming to you, again, with that lens of experimentation.

And the third thing would be, um, just love with technology in terms of using the best tools out there to do the work. Not just, you call three people, and that’s your market research, right?

Alexis Bonnell:

You met a guy and that’s your research. Bryon, jump in.

Robert Fehlen:

Um, I would say they would ask really good questions. So, leaders are remembered by the questions that they ask. And one of the things, so on the industry side, for us and our team, when we’re sitting there and we’re making big decisions about our product or our platform, and if I ask the question, have we asked the customer? And the answer is no, right? Then the meeting’s over at that point, we got to go back to the drawing board, get the person in the room that actually needs to be there.

And I’d say the same thing on the government side. If we’re trying to decide, you know, we know what our problem is, we’re trying to decide what that solution is, and we haven’t started to talk and ask the experts out there what capabilities already exist or where they’re headed, that could really, you know, if we started to do that, that could really speed up, um, where we could go.

Bryon Kroger:

I think, um, one good indicator might be knee-high Chucks, I’ll just say that. No, I think there is something to be said for looking for people who look different. Like, uh, you spot these people really easily in the acquisition’s community. They stick out like a sore thumb. And a few things that I see as like high disagreeableness, um, is definitely something. And these people don’t usually make good careers for themselves in acquisitions. So, you have to find…

Alexis Bonnell:

What are you saying, right?

Bryon Kroger:

No, not you, not you.

And, and, and I would say that, uh, there, there’s an interesting thing here to where John Boyd, anybody, uh, I mean Air Force Hero, right? John Boyd’s roll call to be or to do, uh, you’re looking for the people who don’t care about being somebody. They care about doing something. And those people tend to get overlooked for promotions. You have to go search these people out, but when you find them, they’re worth their weight in gold.

And so, I see one of them here sitting in front of me, uh, was like the best contracting officer ever, uh, in the Air Force. Still is.

Alexis Bonnell:

Call him out, call him out, who’s the best?

Bryon Kroger:

Christina Botello. She’s so mad at me right now. Yeah, she’s awesome. She’s pretty awesome. I think we all love her.

She’s goanna get so many more calls. It’s the end of fiscal year too. She’s like, contracting officers don’t ever want their names.

Um, and, and then, you know, one other thing that is important to look for is, is going back to, you know, you can’t hack a system that you don’t understand. So, these are people that kind of live in both worlds. They’re not like truly the innovators necessarily, but they’re also not the town planners, they’re the settlers. If you’re familiar with that reference, you’ve got your town planners, your settlers, and your pioneers.

We tend to over rotate on looking for the pioneers. We forget all about the settlers, the people that can live in both worlds and actually get things done for the pioneers. So, I would look for settlers.

Alexis Bonnell:

Well, and talk a little bit about, you know, where does that mentality shift? So, as I work with leaders, I think one of the things that I’ve had to navigate too is almost this little bit of willingness to understand there may not be a right answer. We may not be living in a time where there is a right answer, there may be a right for now, right? And that is a very different kind of risk, you know, tolerance to have.

But one of the things I also love is this opportunity to kind of learn by doing, right? So, you know, from an acquisitions lens, does it make sense, you know, to play a little bit, to learn a little bit so that you have evidence? And sometimes we see this, you know, play out with, you know, whether it’s SBIRs or, you know, or kind of small business and really a chance to play.

Um, so talk to me a little bit about that. How do you suggest before we jump to a 50 million, 500 million, you know, kind of contract and we have to move quickly, what does that learning by doing look like?

Dr. Rupa Doshi:

Yeah, I think “move fast, break things” is where the startup world has got to where it has now, um, be it Silicon Valley or other hubs in the country. Um, so the SBIR process is really, I mean, AFWERX has really championed how you can quickly get a, a quick experiment done, uh, without spending a whole lot of money.

The OTA world has also done its bit in terms of really making sure that people can get access to at least trying something out very quickly to see if it has legs, if it has any merit. That I think goes back to the same mentality of experimentation that we were talking about. These people really have to be comfortable with trying something, and it’s okay if it breaks—you learn something from it or something or the other from it always.

Bryon Kroger:

Yeah, I think one thing you’ll have to do if you want to move to a learning model is move to more of a portfolio planning process. So, software portfolios, I think I mentioned Dan Ward earlier, he’s written a lot about this. I’ve had a lot of the acquisition innovation community. One of the things getting in the way of what you would see as traditional portfolio management in commercial is how we allocate funding.

And so, one acquisition hack is pretty simple to say, a little bit harder to execute, especially if you have a legacy program already in place, but you really want to align multiple B-packs under one PE. Some of you might not know those two terms, but it’s just the way that we organize money and acquisitions. It’s really hard to move money between PEs. So, if you’re trying to learn and you’ve got one software over here and another one over here and another one over here, and this one fails and you want to reallocate the funding to the other two successful ones in something like a growth board model, well, today that would probably trigger above-threshold reprogramming, you wouldn’t be able to do it. It’d be really difficult.

So, if you can get your money aligned in one PE, multiple B-packs, now you can have a growth board type model, just like VCs use where I’m doing a bunch of experimentation, this one works, let me move more money into that one and take it from these underperforming assets. And so that would be one hack I would definitely recommend is figuring out how to get your money right.

Robert Fehlen:

I’d just say like on the software side of things, again, breaking it out into stages as well, and having really well-defined metrics for each one of those phases of how are we actually defining success, right? Rather than us just getting a whole list of, here’s the requirements of what it needs to be able to do. And at the end of it, it’s able to do all those things, but nobody uses it. Like, is that actually success in there?

So, us being able to take small phases in there, slowly and incrementally add dollars, you know, find the right vendor along the way for that capability, again, could change.

Dr. Rupa Doshi

Yeah, actually one point to add to that is like, and when you guys are looking at metrics or defining what metric means success, one thing also, and going back to the objectivity characteristic, it’s really good to benchmark that metric against what’s considered industry standard out there.

That you can be led astray by looking at some numbers and a graph thinking, oh, it’s increasing, we’re good, but no, maybe compared to other similar companies or software packages out there, that’s like a 20th percentile piece of success, and that’s not good. You can do much better. So, looking at it from a lens of what’s the benchmark or what’s industry average in this field, will really help you structure those metrics properly where you’ll actually get to the best state of the art as opposed to just something that you think is doing well.

Bryon Kroger:

Yeah. Let’s talk about metrics too really quick, when we talk about learning. Because if you’re gonna say learning is the goal, then you need to have some metrics that show that you’re learning the things that you set out to learn. And so, first and foremost, to be able to learn, I’m a firm believer that you can only truly learn or validate things in production with real users and real data.

So, none of this, like, vendor showed me a demo on a laptop with fake data and then it’ll take two years.

Alexis Bonnell:

No, slideware, Brian, slideware isn’t a real thing.

Bryon Kroger:

Yeah. And it never works.

So instead, the first thing you’re gonna need is a path to production. And so, I would say, like, look into continuous ATO. This is mostly a solved problem, although people aren’t doing it very broadly yet, but the ability to continuously ship lines of code into production and get feedback on them, that opens up your feedback loop. So, this is just the start of the learning process, but you can’t even start learning until you have that.

Now that you have that, you practice user-centered design. And I think there was a big misconception as DevOps started to come into the DOD. Because we call a lot of you fine folks’ operators. So, we said, oh, DevOps, that’s putting developers next to people in uniform conducting missions. That is not what DevOps is. DevOps is about dev and IT ops and putting uniformed coders or putting coders next to uniform military members, like on the job is actually a design anti-pattern. It’s the famous, you know, if I ask people what they want, you would get faster horses.

The goal is to go source problems from you all—you know your problems better than anybody—and then have professional software designers design solutions for your problems.

And when you’re doing that, the measurements that you need to be looking at, or the metrics, are really about deployment frequency and lead time. Those are going to be your leading indicators. And then you can have lagging indicators around some mission metric that matters. Like, hey, let’s reduce the amount of time to produce the master air attack plan by 10%. That would be your lagging indicator. Your leading indicator is what’s your cycle time? How quickly are you shipping new updates to the users and getting feedback on it?

So, I would say, looking at DORA, I wish the Air Force, and the entire DOD would just adopt DORA wholesale—DevOps metrics: lead time, deployment frequency, mean time to restore, and change fail rates. If you measure those four things in tandem, you have a really good learning platform.

Alexis Bonnell:

So, with that, you know, I’m never gonna leave a panel right now without talking about AI, right? And so, one of the really interesting things, um, you know, we’ve seen is just this, this human where human and technology meet, right? And we’re seeing incredible, incredible time savings, speed saving, quality savings when it comes to AI and acquisition.

And, you know, Alexis’s back-of-the-envelope calculation—so this is not, you know, not incredibly evidence-driven, but you know, for me, it looks like if both the vendor partner as well as the government side, we’re really using AI effectively. And we have to, we really have to, you know, we’re talking about shaving off between four to eight months, you know, when it comes to this.

So, when we think about AI, help me think about what are good things people should be doing. And, and Robert, I’m gonna, I’m gonna give you a chance to come in here specifically because one of the things I’ve seen, which I don’t appreciate, is it’s like everyone throws an “AI inside” sticker on it now, right? Like it has AI in it, it, you know, and it, it doesn’t, right?

So please, one of the biggest ways that you can really lose credibility really fast is kind of the “there’s AI in there somewhere” thing. So, you know, that’s one, one asks. I think the second though is really, you know, understanding that great AI, you know, generative AI, it’s really about our relationship with knowledge, right? At speed and scale. It’s about options. It’s not necessarily about answers, right?

And so, as we think about, you know, really the role of humans and acquisition to curate, right? You’re really taking in, you know, what are my options? You know, what are all the things I can be considering? You know, so I guess my second question about AI is, you know, where do people go to look both as far as bigs and smalls?

And what is something where, you know, for those in the audience, you really shouldn’t start doing X with AI until you what?

Robert Fehlen:

Great question. Um, I think back in 2021, you could slap AI in your slide deck and a VC would give you your seed round just about immediately. Um, thankfully we’re getting away from that now.

Um, the reality of like, when we look at AI, it all starts with the data, right? Garbage in, garbage out, always going to be. So a lot of companies right now where we’re seeing some challenges as they’re coming in and trying to cram the product of AI into a space or a problem set or anything like that, we’re not taking a step back and looking at, are we actually designing the tools necessary for the humans that are sitting on the other side of that keyboard to put in the correct data to begin with?

Um, so that’s where we start to create a lot of problems, because without having the right data from the right people at the right time, you can’t generate the insights that you want from AI, right? Or the insights you’re going to get are going to be misleading, going to be incorrect, or just a lot of noise that’s gonna be out there.

So, for us, like we’ve always focused on the cleanliness of like, you have to be able to give the user the space to create, you know, what they wanna create inside the data. And then there has to be some standardized form that we know, okay, this is truth, right?

Alexis Bonnell:

Well, can I prod a little bit more specifically too, and this is for all of you, but I think one of the things is, is like AI is not new, right? There is a lot of great AI out there. And so, you know, all of you kind of in some ways specialize in understanding what are the options out there.

So, you know, I guess help me do two things. One, you know, if someone, someone gets a great presentation, right? And they say, this was really intriguing, you know, how do they look for options? Where might they see, is this the only thing out there? You know, that’s question number one.

And then I think question number two is, what should they be looking for? Or how, how do, how do we suggest making it curious or, and being able to empower curiosity to see what those options are.

The last thing that I would ask you, and as you’re taking all of this in, is not all AI is the same, right? So generative AI or decision analytics is different than wargaming, which is different than C3, CA. And I, I do see a little bit of a trend that somehow AI is, you know, is oxygen.

So demystify a little bit, not only where might you find your options when you get inspired, but how do you avoid kind of thinking this is all one universal thing and not really understanding maybe the, the communities around each of those elements.

Dr. Rupak Doshi:

Um, that’s a lot to unpack there.

Um, so first of all, actually, um, to go back to a little bit of the, the AI, the how, how data is used in AI models nowadays. Um, so definitely, like, there was definitely a time where AI was not that good at dealing with data that might be a little unstructured, but at least on the commercial side, a lot of large companies have realized that over the last few years, that is what they have.

Because people have been putting in unstructured data in different formats. There’s a lot of variability in companies that are large, 10,000, 20,000 employees, right? And so a lot of work has been done to actually make use of that unstructured data also very intelligently with AI.

At the core of it all, most AI models, what they’re really good at doing is pattern recognition. That’s what they are really, really good at doing. And then replicating those patterns in different ways.

And so actually, no two models will ever be exactly the same. Almost, almost never. Absolutely no two AI systems are ever the same. No two applications will ever work the same way. And so it’s definitely onus is on the person looking for this to identify what are the various options out there. And there’s lots of help around there.

So, um, that, that, I mean, I’m sure, okay, in this audience, how many of you have heard of Tradewind AI?

Alexis Bonnell:

Tradewinds?

Dr. Rupak Doshi:

Okay, so 80% of you who haven’t heard of it, you absolutely have to go and watch it. It’s CDAO’s platform that with what they have done is they’ve gotten a lot of AI-powered tools on there that, uh, each one of them has like a little pitch that you can watch, um, and shows you actual use case of that system. And each one of those has gone through a competitive process so that you can actually work with them on an OTA prototype, quick contract, quick experimentation, and it’s all very rapidly done.

That’s one tool out there. Um, we built a tool called Turbo Innovate. That Air Force, uh, basically AFWERX, um, has made it available to all of Air Force. Everyone with the us.af.mil can use it, can log in and use it, which basically catalogs every single innovation and technology across government, industry, and academia all in one place. And we actually use not just generative AI to summarize what’s in there, but also semantic search, which was actually a lot of it, uh, was based on that.

So a lot of people here who have heard about RAG or retrieval-augmented generation-powered AI, that’s kind of how you can get AI to give your response that is super specific with a reference for every single thing that it’s saying. So you can actually have some trust in where it’s getting its responses from.

And I’m sure, um, Robert was gonna talk to you about vision as well, right?

Robert Fehlen:

Yeah, no, I can definitely add in there. I’d say to answer your question too, of like, if you’re on the government side and you have these companies approaching you, right?

Is I, I will also sit up here and say I’m not the expert in AI, right? I have people on my team that are the experts in AI. I, I’m good enough that I got the foundational knowledge, I can sniff out on the backend if you’re just using a plugin versus like, you actually have some secret sauce in here, right?

But it’s also looking to the others in the room that could be experts on the vision side. So our platform, the joint innovation system where we have government stakeholders across every branch of the military working on innovation, process improvement, research and development inside of there, we have people that are looking for projects that are similar to the problems that they need, um, and saying, “Hey, they’re already further down the road than we are. We need to reach out to this team, connect because they have experts on their team.”

So we’re helping, you know, that’s one small way that people are using the platform is to find the experts that are on the government side that sit on the same side of the table, um, to help answer some of those questions.

Bryon Kroger:

Probably have a bit of a contrarian take here, but I think for the most part, uh, I, I wouldn’t be looking to pull AI in until you can reliably ship code to production. And so I think this really like, um, you know, I go around and when I visit most Air Force and Space Force base, uh, like the problems airmen are facing is that they’re working on whiteboards. Their emails don’t load, uh, if they can get their email client open, right? These are the real challenges that we’re facing.

And I think a lot of times, if you are familiar with the three horizons of innovation model, um, most of our programs are in what I call Horizon zero. They’re just zombies that just continue to lumber on, and there’s no horizon one or horizon two that would be things that are generating like mission value today and mission value tomorrow. And then we spend a lot of time talking about AI, machine learning. We don’t talk as much about blockchain anymore, thank God. But, um, and, and so it’s not that AI’s not powerful, like we will need to get there, like to win the next war, we’re gonna need AI.

But before we get to AI, I think we should really, like, you have to prioritize ruthlessly. And I would focus on just getting some of the basics, the fundamentals down before we’re taking half-court shots with AI. I will say though, on the business side, like if I distinguish between mission and business type functions and even some that are kind of in the boundary of those two, I think there’s a lot of room to pull in AI.

We talk about shifting left on things all the time. People don’t shift left. If I ask one of the generals here to shift a meeting left, you can only shift their meeting left if we make space for it. And that’s true of any airman or civilian in here, they can’t shift left on anything until you give them space. And a lot of generative AI in particular, I think can free up a ton of white space for people to start shifting left on high value activities.

So if I were gonna use it, I would just try to make gen AI widely available to the Air Force and Space Force enterprise and just start there. And then now hopefully that frees up time to focus on some of these first class problems we have.

Alexis Bonnell:

Yeah, actually, I love that, uh, because one of the things that I hate the most is toil, right? And it’s really interesting because I think as we go up the leadership ladder, the irony is a lot of the toil is taken away from us, right? And so it’s like, “Well, this is, this isn’t so bad.” And it’s like, “Yeah, because you don’t have to do it,” right?

And so I wanna highlight something for you all. If you haven’t seen this, does anyone know what I’m holding up right now? Does anyone recognize this? Yeah. Okay. If you have not seen this, those are the best $5 that you can spend as a leader.

And what it is, is it is the OSS Simple Field Sabotage manual from World War II. And in essence, in this manual, you know, this is a precursor to the CIA—in this manual, we in essence told our allies, the French resistance and Belgian resistance, et cetera, how to slow down the German war machine, right? How to make it so that you know, it was less effective.

And I’m gonna read you a few passages. Number one: when possible, refer all matters to committees. Further study and consideration attempt to make the committees as large as possible, never less than five. Haggle over precise wording of communication, minutes, and resolutions. Advocate caution and being reasonable. Urge your fellow employee to be reasonable and avoid haste, which might result in embarrassment or difficulties later on. Demand written orders. Bring up irrelevant issues as frequently as possible.

So I highlight that because one of the things that, you know, we do in our team is when the team comes up and they have an idea, they have something they wanna do, uh, two things. Number one, we have a kill bonus. So we actually recognize you and incentivize you for bringing something you can stop doing, right? Because to Brian’s point, you’re not gonna get minutes on mission back, you’re not gonna be able to move left if you don’t get it.

But the second rule we have is that if it’s in this, it’s not how we do it. Like seriously, if it’s in this, if it’s in a sabotage guide, it cannot be the way that we function.

So bringing us kind of to a close in that theme, if we were going to give each of you the ultimate power to kind of hack government bureaucracy, and you were, you know, a king for a day, what would be the thing that each of you would do with those incredible bestowed powers?

Dr. Rupak Doshi:

People are gonna hate me for this, but, uh, force an audit on the contracting officer for every contract that was awarded, uh, to see what kind of market research was performed to come up with the dollar amounts and the vendor that they used.

Alexis Bonnell:

Oh, so you want like an awareness of alternative audit? Okay. All right.

Robert Fehlen:

I would, I would go back to the, um, getting rid of the requirements list when it comes to the, at least the software and some of these things that we need to move very quickly on, and switch that to getting our problem statements out into the industry as much as we can, right? We can’t put everything out there, but as much as we can put our problem statements out and let industry come with the capability, right? A lot of the products that are out there in the industry may not be right, but it could be the capability that they could bring to the table, modify for us.

Um, and then getting really in tune with, okay, we’re coming up with these requirements. What was the actual demand signal? Where did we get that? Was that actually at the end user level that we’re gathering that demand signal to create those requirements?

Bryon Kroger:

I think I would, uh, there, there was a really great book written a while ago, Safi calls Loonshots. Highly recommend checking it out about managing the state change between innovation and steady state or your artists and your soldiers.

I think, um, I mentioned the three horizons of innovation earlier and how we kind of default to zero and, uh, like three, and I think we’re in a phase now where we still use the language of exploit. There’s an explore phase and an exploit phase. We use a lot of the language and thought processes and processes for exploitation. This is where you hear people talk about efficiency. “Oh, there’s 20 software factories. We gotta, we gotta, we gotta have one. We can only have one. This is clearly inefficient.”

We are in the explore phase where we’re still trying to find efficacy, and you have to find efficacy before you switch to efficiency. Efficiency without efficacy is waste, right? You’re doing a thing really well that shouldn’t be done at all. And so I would change the incentives around programs that are delivering software to what I said earlier.

I don’t want to like cost, schedule, performance, uh, earned value management, like all of these traditional incentive structures that we have that go all the way into OPR, EPRs and civilian rating systems and everything else. I would get rid of all of that in favor of production.

First of all, like you have to get to prod in 180 days or less. That’s actually like a brand promise we have at our company. And then from there, you need to achieve weekly software delivery. And if you do that, again, that’s the leading indicator. You still have to get to mission efficacy, but that’s the leading indicator for your ability to learn quickly. And we have a lot to learn, and we have to catch up real quick.

And so rebuilding horizon one and horizon two, I think is a matter of re-incentivizing how we measure people on software programs and their delivery.

Alexis Bonnell:

Let me ask you one, since we have, you know, about two minutes left and I wanna get a little bit of therapy to wrap this up, right? There’s gotta be something in it for me on this. So, uh, my question is this.

One of the roles that I get to play is authorizing official. And I want, you know, one of the things I’ve learned being kind of new is that, you know, we often will talk about, you know, contract officers need to do this or things like that. But I really am honestly trying to take a look, you know, from my therapist, and I’m trying to say, how do I show up as well and as powerfully and as positively in this process?

And, you know, it’s hard because there isn’t necessarily always a lot of incentive to take risks. But this question is, how do I become, instead of the villain of every story in the AO, how do I become the hero? So if you don’t mind, in the last few seconds, tell us, you know, this, this point where we hit that very end of innovation right before it really gets to breathe the type of life. What is your advice for, you know, for that role, for the AO role, and how to show up in a way?

Dr. Rupak Doshi:

Um, clarity. I think that’s the one thing that we often find that is just impossible to get information. I mean, even getting connected to the AO is months and months of time that it takes. Um, so even if you can delegate that clarity of information to other people, people who can do it on your behalf, that would be really, really helpful. Like every organization should have a POC who has a direct line to you that knows exactly what the required, uh, metrics are to hit to get further along in your ATO process.

Robert Fehlen:

Yeah, I would, I would say similar, um, availability, and maybe you as the AO it’s not your availability necessarily, but you have that point of contact, that, that wall in between of sorts, um, to just get that clarity of like, where are we even at in the, in the process on the way can help tremendously.

Bryon Kroger:

Yep. So like the hardest job in the Air Force, uh, I would maybe—so I’m in a, being the hero doesn’t necessarily mean you’re gonna be liked, so I’ll have a little bit of a contrarian take here again. And that’s just to say that, uh, you have to do what’s required.

And I think the more, as leaders, particularly AOs, can really get their people, uh, one, focused on what is required and bought into it—I mean, look, I would like, if somebody came to me and they’re like, “Hey, I’m trying to, you know, fit into my wedding dress in a year, what do I do?” I’d love to say like, “Eat all the cake you want.” But instead I have to say like, “Hey, you know, here are like five things I think you need to do.”

There are just so many things in the enterprise that are required. I’m not even talking about bureaucracy. And I have this refrain, and it’s, it’s something that I talk about with my team—we have like this internal podcast, I say it a lot—and that’s, I will do what is required. And I don’t mean crappy requirements process just to be clear, things that are actually required to get things done.

Like if I want to be able to do a hundred pushups, I’m gonna have to do a lot of pushups every day. If I wanna get through the ATO process, there are just things that I have to do. And look, there are things I would change about NIST RMF and how the ATO process is, but in general I can get through it without impacting my DevOps speed. I can go fast and do RMF; we’ve proven it over and over again, and it’s just a matter of doing some work.

And I think most of what we see in the community today, like, “Oh, there’s no reciprocity.” If I were an AO I wouldn’t grant reciprocity to most of the packages I’ve seen. And so do the work. I would constantly be telling people like, “Here’s what the work is: clarity and do the work.” And if you, if you do what we said that you needed to do, you will get an ATO, and you will get it quickly. And if you can follow through on that, you’ll get people to do the thing that they’re supposed to do, and we’ll have much better ATOs.

---

This transcript was auto-generated and may not be 100 percent accurate. The source audio and video can be accessed above.