Last year Congress allocated $48 billion, as part of the Infrastructure Investment and Jobs Act, to broadband (internet) access. But more than the money, we need to know where we need broadband before we can build it. Our guest for this episode, Sarah Oh Lam, talks to us about the Tech Policy Institute’s (TPI) Broadband Map Project.
Follow Sarah on Twitter: @sarahecon
Related Topic Research/Articles:
Comments filed with NTIA
State of Broadband Report
TPI Broadband Map
Last year Congress allocated $48 billion, as part of the Infrastructure Investment and Jobs Act, to broadband (internet) access. But more than the money, we need to know where we need broadband before we can build it. Our guest for this episode, Sarah Oh Lam, talks to us about the Tech Policy Institute’s (TPI) Broadband Map Project.
Follow Sarah on Twitter: @sarahecon
Related Topic Research/Articles:
Comments filed with NTIA
State of Broadband Report
TPI Broadband Map
Disclaimer: This transcript is auto-generated and has not been thoroughly reviewed for completeness or accuracy.
[00:00:00] Gus Herwitz: This is Tech Refactored. I'm your host, Gus Herwitz, the Menard director of the. The governance and technology center at the University of Nebraska broadband is a pretty important topic nowadays, and it turns out it's actually been a pretty big topic in, uh, universal service or telecommunications policy for a couple of decades.
For more than the last two decades, the government has invested over 5 billion a year in what we call universal service programs, and just this past year, Congress has allocated an additional 48 billion to broadband infrastructure as part of the infrastructure investment and Jobs [00:01:00] Act. But even more than money, it turns out it's really important to have a sense of where we need to build broadband before we can build it.
Really having a picture of maps of where broadband is and isn't available can be a really big problem when it comes to universal service and broadband deployment. Today we're joined by Sarah Oh Lam. She is a senior fellow at the Technology Policy Institute, where, among other things, she works on a project to map where broadband internet is and isn't available.
She has both a law degree and a PhD in economics from George Mason University and a degree in Management Science and Engineering from Stanford University, and I'm happy to say she's also a visiting fellow here at the Governance and Technology Center. Sarah, welcome to Tech Refactored.
[00:01:44] Sarah Oh Lam: Thanks Gus. Thanks for having me.
[00:01:48] Gus Herwitz: So you, you, uh, heard how I introduced this topic, uh, in the introduction there. Um, can, can you just, uh, start by telling us a little bit about, uh, this broadband mapping challenge, [00:02:00] um, why it's so important, and frankly why it's so hard to do.
[00:02:05] Sarah Oh Lam: Sure. Well, at the Technology Policy Institute, um, we're looking at the broadband maps that the FCC put out and we actually started building our own broadband map.
Um, so we have a TPI broadband map, um, that is a web app that loads up all the FCC form 4, 7, 7 public data sets, the Universal Service fund data sets. Um, and we've been analyzing that data and the layers. And have come and, and found, you know, firsthand limitations of the mapping data and, um, how, how to improve analysis of the data.
[00:02:44] Gus Herwitz: So you, you're, you're taking us straight into the depths of, uh, FCC land. Um, you, you mentioned FCC form 4 77, um, and also, uh, universal service maps. Um, can, can you, I, I hate to ask this and I apologize to [00:03:00] our listeners, but can you explain what FCC form 4 77 is and, uh, also, I guess what it.
[00:03:07] Sarah Oh Lam: Sure. Um, so I think a lot of people have heard the term broadband map.
And so that's basically a map of where broadband connections are in the United States. Well, then you peel back a layer and you ask, Well, how do we have the data, um, and provider submit self report? Their deployment maps to the Federal Communications Commission in a form called Form 477. Um, this data collection by the FCC, um, has been going on for many years, and that's pretty much, um, the standard data set that we have right now for where, for what speeds and where, um, broadband is available.
Now there are many limitations to the Form 4 77 data because, um, of the methodology that the FCC has put on it, um, basically the limit is, you [00:04:00] know, if a provider serves like one user in a census block, um, that whole block can be called served. Um, and so, The geographic area is a little bit too big, um, and that, that creates a little bit of distortion.
Um, and it's about the Form 477 data. Um, but that is kind of the best data we have right now. Um, state by state and private data sets are, are more, um, exacting and under the broadband data Act. Um, the FCC will soon have even better data. In, in something called a broadband fabric data set, which is supposed to have, um, house or building level, um, data on connectivity.
[00:04:49] Gus Herwitz: Yeah, so the, the history of Form 477 for, for anyone who's, um, uh, familiar with discussions about broadband mapping, uh, you, you've probably heard of Form [00:05:00] 477 and thi- this is, uh, a puzzling topic for a lot of folks who aren't really, uh, entrenched in these topics because you'd think. Why doesn't the FCC know?
Why doesn't the FCC have this information? Um, it turns out that Form 477, uh, was created, uh, in, uh, the late 1990s. Um, uh, I think it 98 is when it was put in place and the purpose was to track where. Uh, new entrance into the local telephone market. We're starting to offer telephone service. And at the time, the, the question was, did you offer any service in that area?
Because if so, uh, that, uh, affected, uh, competitor's abilities to, uh, compete with you, Um, what the FCC would let them do. So, form 4 47 7 wasn't intended for, uh, broadband mapping purposes, but in the, uh, uh, early 2000. That was the only form of data that the FCC had been collecting. So it's [00:06:00] what we continue to use and it, it's kind of, uh, it's not even kind of, it's just mind boggling that this continues to be, uh, our, our primary, uh, a data source.
But as you know, Sarah, uh, uh, the, uh, a broadband data, uh, act, we've got new statutory authority and new, uh, statutory mandates for better maps coming into place. Um, So can you, uh, tell us a a bit about what, uh, you at TPI have been doing with, uh, broadband maps and what you are, uh, adding to, um, or, uh, uh, doing differently than just, uh, let's look at form 477.
[00:06:32] Sarah Oh Lam: Sure. Um, so we're economists and we ask and answer policy questions. And so that's a little bit more involved than looking at a map. Um, and so a lot of the policy questions are like, where should subsidies. And that kind of question, um, requires a little bit more analysis than finding, um, unserved areas in that form 477.
You kinda [00:07:00] wanna include demographics, You wanna include other information like subsidies, currently existing subsidies. And so right there you have three layers of data that you wanna compress down and, um, you wanna analyze it at a census block, um, or county or district level. And so right there you have a lot of number crunching.
And in our experience, we would load up these separate data sets into a platform like data, Um, and we'd have to do these joins and matches. Um, it would be time consuming, um, cuz they are at big data sets, so it would take maybe a few days to just load up all the data sets and then, Another few days to figure out, well, which geographic area do we wanna look at?
So after doing this manually, um, over and over and over again, we're like, we should just build it into one app where we can keep the data sets in the [00:08:00] cloud , um, put 'em into like a SQL database, and then, oh, we can add some Python code and like, oh, we can add an interface, um, oh, and then we can like look at it on a map.
Um, and so, We built our app from the perspective of economists like running, crunching the numbers behind it. Um, Whereas, I mean, I, we hope that other maps, um, have that analytical capability behind them. Um, but so far a lot of the state broadband maps are, um, are really beautiful images. , like high resolution arch gis, like maps, heat maps, um, but there isn't that much like analytical capability on the maps.
Um, so we're building that computing power on the back end in our map. Um, Yeah, but just from a policy perspective, we've realized that a lot of folks state [00:09:00] legislators press, they want, you know, analysis done, like answers. Um, and so we're do that analysis and the, you don't need to really do that whole like, Ole every time, um, you get a question from a state legislator, um, we're doing it automatically and automating it, Simplifying the, the calculations.
[00:09:29] Gus Herwitz: Yeah. So there, there's a important business lesson here, though. Uh, I, I guess you're not doing this commercially. Um, but in many ways this is something I, I, I just love talking about, uh, thi this is what made. Such a successful company. One of the things that they, uh, uh, did is whenever they needed. Function or some feature internally, some process internally, they said, Hey, let's not just do this on a one-off basis.
Let's build this as infrastructure, as a platform that we can use. [00:10:00] And if we need it, probably other companies do so we can start selling it as a service. . Um, so you guys, uh, kept going through this process over and over and over, and you, uh, uh, turned it into an internal. Uh, that's, uh, a turnkey and now it's available online.
And we, we should say, uh, tpi broadband.com. Uh, that's, uh, where the current version of the map is. It does say it's still a beta, um, uh, version, but, uh, I've just, uh, pulled it up. Uh, here and clicking on Nebraska, for instance, uh, we have information, uh, the average maximum available data speed, a download speed, 802 megabit per second.
Uh, we do have a fair bit of gigabit fiber, uh, in, uh, uh, the, uh, more urban parts of the state. Um, average available max upload speed. Percent of the state with 4G coverage, 5G coverage, and then you get into, uh, various of the data sources. So, uh, Okla, ACS [00:11:00] could keep scrolling down. I'm not going to, but, uh, the, uh, thing that I actually want to focus on.
So first just a, a interesting, uh, useful compilation of data there, but you go up to data. And you start FCC form 4 77. But then the American Community Survey, acs, uh, the OLAS speed test, microspo- Microsoft's speed threshold data from NTIA, uh, uh, the FCC's Rural Digital Opportunity Fund and Emergency Benefit Broadband Benefit Programs.
So, uh, how, how, why, Actually, I'll start with this. Why haven't people. Compiled this information this way previously, um, and, and made it available in this way.
[00:11:45] Sarah Oh Lam: Um, yeah, I think that's a great question because so many people ask these questions and wanna know answers about, you know, where is EBB money going?
Who is adopting? Broadband. I, I think it [00:12:00] is a little bit niche, uh, . There aren't that many people who want to do the econometrics and it's statistical To do the analysis properly, you need to, you need to do statistical analysis. You need to. Control, you know, to compare geographic areas you need to control for income control for other demographics, age to, to really compare like one area to another.
I think. I think we're there are other folks doing similar work? So our next report is going to be a comparison of broadband maps and there are a few proprietary platforms that offer a lot, all these layers and then some comparison or analysis. I don't know. I don't know why. I guess there's no demand for it.
[00:12:48] Gus Herwitz: Well, I, I, I disagree and think you're selling people short, Sarah, perhaps, uh, um, uh, idiosyncratic, but you've got a, a definite customer base of one here, at least and I, I say that [00:13:00] jokingly, but we hear so much discussion about broadband deployment and universal service and the importance of broadband. And I, I can't imagine frankly, going a single day without hearing some discussion of these issues on the news, but we know.
These are important needs, important topics, but without the data to feed our understanding, both of where broadband is available and it isn't available, it's really hard to have a meaningful discussion about the, the scope of, uh, these challenges and to address them. What, one more question about the, the process behind the map and then turning to a bit of a more substance about what lessons we can take away from the map.
You, you're pulling in all of this information from different data sources. What, what other data sources are on the horizon that you might be able to pull data in from? How complimentary is this to other mapping [00:14:00] initiatives? You say you're doing this comparison between various, uh, available mapping platforms and the mapping sources.
Um, how close are we to. Actually having a relatively complete picture of broadband availability.
[00:14:17] Sarah Oh Lam: The more that we are working on this app, the more we feel like, oh, this is a product that we'll be using for, you know, a long time. How complete is the data? Well, For instance, the mobile. The mobile Form 477 data.
The deployment maps, the polygons are actually really complicated. They're very custom. So for us to put load that into our map is taking some time.
[00:14:41] Gus Herwitz: So polygons, you, you mean the, These are actual polygons, geographic shapes that represent areas on a map.
[00:14:48] Sarah Oh Lam: Yes, like, uh, GIS, the geospatial, I guess protocol or how, how you describe an area on a map.
Polygons are, the data field is a [00:15:00] list of vertexes and, and the mobile maps are like really intricate. So we actually ran up like a huge Google Cloud bill one day because we had some code like running to analyze, um, Analyze those shapes and it just like exponentially took off. We doing intersections. A was just ran this loop.
We didn't realize it was so exponentially complex. So I think your question was "how close are we?" Well, as the data gets more high-resolution, like the calculations get a little bit more complicated, but the FCC actually provided some intersections already and I think that. Is a point about the government's role in providing good data or like doing that kinda data analysis. I wish that USAC, the, the cooperation that handles the Universal service Fund would do a little bit more with their data.
They have so much [00:16:00] data that just that one institution could do to help researchers on the outside know where the money's going. And so the burden. Should be on the least cost provider of data, um, analysis and, and kinda a centralized place. And a lot of that is the government. So the fcc, the usf, they could do a lot to make better data available to, to US researchers on the outside.
And I think, I guess, How far are we? Well, a lot of it also comes in like how clean the data is. Like the emergency broadband benefit data. It's at a zip code level. To provide anonymity. Some of the zip codes are at three digits, not five digits, because if it was as specific as five digits, you could maybe identify households in, in sparsely [00:17:00] populated areas.
And so in some areas they only provide three digits. So things like that, I think our progress for, for putting together a good map, Other things, eRate or, you know, naturally the money goes to a school, but you can't really track like which households like the students are coming from. So it, it's basically a tracking problem.
[00:17:28] Gus Herwitz: The, there's such a important point there. The, the government. Is very frequently in the best position to have this data and to share this data and do useful things with this data. And they, they often just don't, which is a, uh, a constant frustration for researchers. It, in fairness, it's typically not in federal agencies statutory mandates to make this data available, especially to do, make it available in any, uh, specific form [00:18:00] and.
Expensive and costly and time consuming to do that. So that, that's a, uh, great background point, uh, for anyone working in these areas of research, finding ways to get access to and work with and improve access to this data, especially if you're a government agency. Uh, and I also just want to note, uh, uh, you mentioned Sarah, uh, USAC the Universal service administrative company, which is a, uh, a public interest corporation or a corporation created by the government. Folks might be curious to just have some sense. You said it's this corporation that runs this federal program that, that seems a little odd. The government actually creates companies frequently to administer things like that.
So, uh, an interesting element of our bureaucracy. Well, We should take a brief break, Sarah, and when we come back, I'd love to, uh, get into some of, uh, the data that you're finding or the lessons that you're finding or policy directions, uh, that we might want to go based upon the mapping work [00:19:00] that you are doing at TPI listeners.
We will be back with more in just a moment.
[00:19:07] Sarah Oh Lam: Momentum it's building at the University of Nebraska Lincoln with game changing work in precision agriculture, nanoscience and digital humanities. We're unlocking mysteries in brain research, solving the impossible with remote surgery using robots, and we're creating bold futures with world leading research in early childhood education.
We don't slow down and we are not letting up. We are Nebraska.
[00:19:36] Gus Herwitz: We are back talking with Sarah Oola about, uh, the TPI broadband mapping project that she, uh, is helping to develop with the Technology Policy Institute. Sarah, I, I know that one of the things that you are working on with this data is something that you're calling the broadband Connectivity Index.
Can you start, uh, just telling us, uh, what this index. [00:20:00] Sure.
[00:20:00] Sarah Oh Lam: So Scott Walston, he, he's been doing most of the brain work behind the broadband connectivity index, but the idea is that connectivity is not just download and upload speed. And so a lot of policy talk goes around, Oh, broadband should be defined by 25 3 or a hundred or hundred 20, meaning a hundred megabit.
Download and 20 megabits upload speed or symmetrical upload download speed. The idea is, oh, like we should set our policy goals higher, that people need faster broadband. Well, it's not just speed that matters for connectivity, though there are a lot of other dimensions, um, for a good connection like latency.
What else? The type of technology and then you also wanna know, Like, what is kind of the [00:21:00] area like that you're in comparing connections in, in the area that you're are, is, is one area surrounded by a lot of connectivity or is it surrounded by like really no connectivity and so all these inputs go into an index that we've created, and this is very typical in other.
Kinds of policy analysis and financial analysis too. You hear of all sorts of innovation indies and growth indice, and so we wanted to do the same thing about broadband connectivity. And this index is meant to be helpful for policy makers to know where to send subsidies, where to concentrate their attention.
Most state lawmakers are gonna have to answer and providers are gonna have to answer the question. Which areas are unserved and what, where should subsidies go? So that definition of unserved could vary state by state under the infrastructure monies. [00:22:00] It doesn't have to be 25, 30. It could be higher or lower, or it could, it could be something else.
And so we created this BCI to, to add more information to that conversation.
[00:22:14] Gus Herwitz: Uh, are there other interesting lessons that you're learning as you start to, or continue? I know you've been working with this data in, uh, a less map like forum for quite some time. Uh, are, are there lessons, uh, for policy makers that you're finding in this data or just surprises?
[00:22:31] Sarah Oh Lam: So the data can be a little bit, you know, overwhelming if you print out like a report of what's the broadband state of broadband in my county, because you wanna know all sorts of demographics about a county, how many people, how many households, how many children, What's the income level? What are the jobs like, what's the, what providers are there?
What, what speeds are available? How many subsidy, how much subsidy, dollars are going to the schools there? And so, [00:23:00] I think what state legislators want to know are very simple things like where, you know, which areas should we concentrate on first? How should we rank the ar, the counties in our state? How much money should go to each, um, county, Which providers can we trust to build out projects?
And so, While the data is complicated, the answers are I guess, pretty simple in the end, which brings to, brings to other points that we make that we about just policy recommendations for the states and on distributing, you know, 40 to 48 billion.
[00:23:50] Gus Herwitz: So the, the ntia in the ia ja, this recent infrastructure investment legislation adopted by Congress, uh, better known as the, [00:24:00] the infrastructure bill is getting, uh, 48 ish billion, uh, which is supposed to be distributed, uh, in grants to states.
And this is different approach than we've taken historically to universal service. Um, the FCC has historically focused on giving money to ISPs or telecommunications companies in exchange for promises to offer service. Uh, that program has changed and evolved dramatically, uh, especially over the last, uh, decade or so, moving away from the, the traditional.
Telephone companies and allowing many more entrance and technology into the fray. But the, the NTIA approach is much more state focused, um, and state driven as opposed to provider focused and provider driven. So what, what are some of the recommendations that you're making for NTIA to look at as it, uh, embarks upon this process?
[00:24:55] Sarah Oh Lam: Yes. You know, half of these points are related to data like we talked about earlier, [00:25:00] but one thing is to require the states to make public all information on applications. So as grant applications come in, even though they're not funded, it would be really helpful for that information to be made public so that researchers like ourselves can see Are the states using competitive bid bidding to compare bids?
What prices are providers offering? Where they offering to build out? So that transparency would be really important. Normally, states will make public the applications and the proposal documents for grantees that they fund at the end, but a lot of times they don't make public the the grants that are unfunded.
So that full data set would be really important to know the supply of BroadB. Available. We also recommend that the states conduct monthly progress reports based on some standard objective like [00:26:00] metrics that allow the NTIA to make cross-state comparisons of progress. So you can imagine every state might report back to NTIA with their own custom metrics.
And then a researcher like myself would not be able to compare money going to one state with another. So that's one just obvious. And the reason why it's important to have these standardized like metrics is because, We need, we want to learn from this huge outlay of money. We wanna know which projects were more successful.
We wanna track and see like which states like invested their dollars invested, distributed their dollars better than others, where outcomes improved more than others. So, and it's a very simple thing for that NTIA could do to ask the. To fill out a standardized form in reporting back with [00:27:00] meaningful data points, and I don't think it's too much to ask of the states to do that.
A few other points that we recommend, some long moonshot requests NT could offer a platform. Is like off the shelf procurement bidding system. So states that don't wanna create it on their own could just use the NTIA's software that would require NTIA to like come up with an auction methodology. But if NTIA could do it and the states want to opt in, you can imagine there'd be economies of scale and cost savings for all the different state.
[00:27:42] Gus Herwitz: We should say there that in, in recent years, that's the approach that the FCC has used to allocating. Um, its funding what, what we call an auction or in some cases a reverse auction. And it's a complex thing designing these auction systems. The FCC goes through training with [00:28:00] all the participants involved with it.
It's a. Fascinating and complex software platform. And there there's a lot of knowhow and a lot of lessons that have been learned there. Um, and a, a lot of states are talking about using competitive bidding or auctions to allocate this funding. And you, we don't want them relearning all of the lessons.
And frankly, that means making all the mistakes that the FCC has already learned and made in now 30 ish years of using auctions to allocate various resource. So, yeah, let, let's, uh, get the nt I doing some standardization and offering auctions as a platform to, uh, the, the states and hopefully working with the FCC to bring that knowhow to the states.
[00:28:45] Sarah Oh Lam: Yeah, and I listened to the oversight hearing two days ago with the new N Administrator Davidson. You know, while it's hopeful, sounds like they're doing, you know, [00:29:00] learning from and getting ready to have a really good, um, Process of reviewing state broadband plans. You know, I, I just hope that NT I can do the best they can to repeat, to prevent duplicative work across the states.
I can just imagine that they'll let the states each do whatever they, you know, see fit and then it'll be hard to compare later, but hopefully NT can. Have foresight, especially for program evaluation. Like we really want to study how the money spent, not just spend it. So that kind, thinking ahead and planning for, for program evaluation is really important.
[00:29:52] Gus Herwitz: And we, we should say, BTOP was the Broadband Technology Opportunity Program, which of course, uh, [00:30:00] was created during the, the last major national crisis. Um, the, uh, uh, crash of, uh, uh, 2008 2007 through the American Recovery and Reinvestment Act. And that was a $4 billion program administered by, uh, NT for broadband build out.
And there, there's- just an important point here that you're, you're making Sarah, that we have NTIA working here. Uh, the FCC works here, uh, USDA administer programs here states. Many states have, for a long time had their own universal service programs and NTIA is going to effectively be fueling the development of many more of those.
Pro we, we need coordination. We need to make sure that we're not, uh, uh, duplicating efforts and even, uh, more important and concerning making sure that different programs aren't working at competing to competing ends and really creating conflicts with, uh, one another.
[00:30:59] Sarah Oh Lam: [00:31:00] Yeah, so I wrote my dissertation on Btop and, um, I studied the, the middle mile portion of the program, and I were selected applications that weren't necessarily cost overruns.
I, I was in the cost overrun literature at the time, but there were also like, Not that many competitive bids. The prices that were offered by the providers were kind. Not, not necessarily the most. There was no auction. And so it was all a grant review process and there was little evidence that there was any price, um, comparison.
And, you know, out of urgency, the money is shoveled out the door. Like grants need to be, um, written and the money needs to be spent. But [00:32:00] with that comes, you know, I guess waste or, um, less optimal outcomes. Um, and that's where, you know, you can compare it with the fccs USF program, which distributes 10 billion annually using auctions, learning from the past, hopefully.
And so you hope that the states can learn from the FCC. Um, , but maybe not . I mean, it, it might end up that, um, every state ha comes up with a broadband plan and, and they learn. But I, I should be more hopeful.
[00:32:45] Gus Herwitz: I have to say, Sarah, I've, during this conversation, I've actually had, uh, the DPI broadband map application running in my web browser, and I've just been looking at data here, state by state, and there, there, there's a whole lot [00:33:00] of interesting stuff here as there you always find when you have a presentation of data and a good way to, a good tool for exploring that data. Um, so I'm looking forward to jumping in here and continuing to see what all I can find and discover, and I'm certain that this will be a useful tool for policy moving forward. So thank you for this work and thank you for joining us today for this discuss.
And for our listeners, thank you as always for joining us as well. I've been your host, Gus Herwitz. We hope that you enjoyed this episode of Tech Refactored. If you want to learn more about what we're doing here at the Nebraska Governance and Technology Center, or you want to submit an idea for a future episode, you can go to our website at ngtc.unl.edu, or you can follow us on Twitter at UNL underscore NGTC.
As always, if you enjoyed the show, please don't forget to leave us a rating in review wherever you listen to your podcasts. Our show is produced by Elsbeth [00:34:00] Magilton and Lysandra Marquez and Colin McCarthy created and recorded our theme music. This podcast is part of the Menard Governance and Technology Programming Series.
Until next time, keep digging into those maps.[00:35:00]