LinkedIn Recruiter wants to help you find women – Protocol

Share on facebook
Share on twitter
Share on linkedin
Share on whatsapp
Share on pinterest

Last month, LinkedIn launched a new feature called “Diversity Nudges” to show recruiters how to widen their search and recruit more women.
Leaders sometimes say underrepresented talent is “hard to find.” Usually, they’re just looking in the wrong places.
Tech leaders have made their commitment to DEI loud and clear. Meta, Google and Apple publicly back affirmative action, cybersecurity companies are trying to close the talent gap by hiring diverse employees for entry-level positions and some of those meticulously tracking their numbers have met some of the goals they set for themselves. But tech’s progress in hiring workers of color, women and other underrepresented folks is still incremental.
One excuse leaders give is that underrepresented talent is “hard to find.” Usually, they’re just looking in the wrong places.
Last month, LinkedIn began rolling out a feature called “Diversity Nudges” for LinkedIn Recruiter, the paid plan that companies use to find and manage candidates. The new update is designed to challenge that “hard to find” myth and help hiring managers find a more diverse group of candidates. According to LinkedIn, the feature will be available everywhere in the coming weeks. Right now, the feature only works for gender diversity.

If a recruiter’s search results skew either too male or female, the nudge feature will recommend ways to broaden the pool of results. LinkedIn says that if less than 45% of the talent pool is male or female, hiring managers will see a banner in their search urging them to change the criteria. The nudges also recommend certain locations, skills and companies that will most impact the hiring search. For example, adding candidates from a particular location or with a particular skill might increase the percentage of female candidates.
“It’s not an extra step that we’re asking recruiters to take,” said Jennifer Shappley, VP of global talent acquisition at LinkedIn. “It’s built into the day-to-day activities of a recruiter.”
Diversity Nudges will direct recruiters on ways to appropriately widen their talent pool.Image: LinkedIn
The goal is to help hiring managers expand their preconceived ideas of what makes a candidate qualified. Shappley noted that women tend to emphasize soft skills and men tend to emphasize technical skills. A nudge might tell recruiters that including soft skills in their search criteria will amass more female candidates. Ideally, that knowledge will stick with them.
Mandy Price, co-founder and CEO of Kanarys, a diversity, equity and inclusion technology company, told Protocol that “while many organizations claim there’s a pipeline problem when they can’t find diverse talent, we know that’s not the case because talent resides everywhere.”
LinkedIn infers gender through the information you provide in your profile and pronouns that others use when they endorse you. According to its website, LinkedIn does not infer gender through profile photos. Some people tell LinkedIn their gender in an “anonymous manner,” Shappley said, as the information is not public-facing. LinkedIn can also infer gender via LinkedIn identity-based groups listed in profiles. Anyone using LinkedIn can read how the company uses their demographic information, update their own and remove that information later if they so choose.
Diversity Nudges are limited to gender right now, as the company doesn’t have enough data yet to identify race, age, sexuality or whether someone is nonbinary. “Once we have sufficient self-ID data to have accurate sample sizes, we’ll also be able to expand our insights to race, age and more,” a LinkedIn spokesperson told Protocol.

The idea is that Diversity Nudges would make it that much harder for leaders to lean on the “they’re hard to find” defense, and is one more way to hold companies accountable to their DEI promises. Many big tech companies invest in elaborate tech that uses machine learning to root out recruiting bias. But that doesn’t mean they shouldn’t try the simpler solutions too.
Recruiters might not be aware that their approach to the day-to-day candidate search is actively impeding their diversity goals. “It is really hard to get people out of the routine that they have become used to,” Shappley said. “Being able to connect folks to that outcome they say they want with their daily behavior is really important.”
Two recruiters who spoke to Protocol were interested in the idea of the Diversity Nudges, but pointed to another feature of LinkedIn Recruiter as potentially even more useful. Both Kindall Carlson, senior technical recruiter at DailyPay, and Dan Logan, senior director of global recruiting at Beamery, said the tool that allows recruiters to hide photos and names in their search could do even more to eliminate unconscious bias.
Price from Kanarys called the Diversity Nudge “a great first step.” But if companies really want to fix their diversity problems, they must look within. “They need to examine their entire policies, practices and procedures to eliminate bias — not just change how they search for talent on job boards and other online platforms.” Price said this includes “standardizing interview questions, creating diverse interview panels, reworking job listings to include inclusive language, shifting to skills-based hiring and more.”
Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She’s a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school’s independent newspaper. She’s based in D.C., and can be reached at llawrence@protocol.com.
Megan Morrone (@ meganmorrone) is a senior editor at Protocol Workplace. Previously, she was a senior editor at OneZero, Medium’s premier technology publication covering technology’s effects on people and the planet. Prior to that, Megan hosted and produced several popular video podcasts at This Week in Tech and was an on-air contributor for The Screen Savers, a daily live television show about computers. She is currently the mother of three teenagers and the president of her dog’s fan club.
Beta Technologies’ charging network offers a way to power up both electric planes and vehicles, and a new app will help improve access.
Beta’s ALIA-250 electric plane uses “electric vertical takeoff and landing” technology and has a range of up to nearly 290 miles.
Lisa Martine Jenkins is a senior reporter at Protocol covering climate. Lisa previously wrote for Morning Consult, Chemical Watch and the Associated Press. Lisa is currently based in Brooklyn, and is originally from the Bay Area. Find her on Twitter ( @l_m_j_) or reach out via email (ljenkins@protocol.com).
Electric aerospace company Beta Technologies is working to build a network of chargers that could power its forthcoming small electric planes as well as ground-based electric vehicles.
The network is still in early stages, with just 10 on the ground and an additional few dozen in the permitting or construction stage.
Beta is also launching an app in an effort to get pilots and drivers to its chargers, the company shared with Protocol. The current version of the app is quite simple, providing just enough information for users to find a charger and start a session. But Chip Palombini, who heads product for the company, said the goal is to eventually build it out in response to customer need; while Beta’s planes are not yet in the hands of customers, a growing number of chargers are ready and waiting for both cars and eventually planes as well.
The Burlington, Vermont-headquartered company has two prototype planes that are still in the testing stage, and it plans to seek Federal Aviation Administration certification by 2024. The planes use “electric vertical takeoff and landing,” or eVTOL, technology, which removes the need for seeking out a landing strip, and have a range of up to nearly 290 miles. The company says its planes will be able to carry 1,400 pounds of cargo or six passengers depending on the configuration. UPS, the Air Force and United Therapeutics all have purchased agreements or established partnerships with Beta.

The process of actually getting electric planes to customers is an uncharted one, though, given that no electric aviation company has received FAA certification for commercial use. Heavy batteries take up valuable payload and limit the range of electric planes as well. A recent report from the International Council on Clean Transportation found that improvements in battery storage technology are still needed to make using electric planes “feasible,” especially given the requirement that commercial planes have reserve fuel.
In addition to building out its charging network, Beta is launching an app in an effort to get pilots and drivers to the chargers. Image: Beta
Nathan Ward, who leads Beta’s work on its charging network, said the company hopes its chargers will spread out so that they can allow electric planes to go anywhere they need. In the short term, however, the focus is on building out a network along the East Coast in locations where organizations that have made purchase agreements operate.
For most of those customers, the aircraft is a small part of their process of getting cargo or people from point A to point B. Beta’s ALIA-250 prototype is designed to replace the short-haul trucks that run on fossil fuels and currently dominate the cargo landscape. The company is based in Burlington, Vermont, and sees connecting it and smaller cities like it as a way to ease supply chain issues. Its charging network could be a key piece of making that vision a reality.
The ALIA-250, which was unveiled two years ago, has primarily taken short flights, though the company recently completed a 1,403-mile journey from its testing site in Plattsburgh, New York, to Bentonville, Arkansas, and back, relying primarily on its own 10-charger network. While the trip dealt with weather-related delays, it took 704 minutes of flying time to complete, over the course of a week.

In its all-electric-delivery vision of the future, Beta is betting that its plane-EV combo charger will appeal to customers who will have on-the-ground transportation waiting on the tarmac when planes arrive; as delivery vehicles are increasingly electric as well, the company assumes customers will want to simplify the charging process as much as possible. When UPS announced its purchase agreement for 10 of Beta’s planes last year, the logistics company added a reservation for charging stations as well.
Beta raised $375 million in series B funding earlier this year, adding to the $368 million series A round it announced last year. The funds come from heavy hitters, including Amazon. While there are still numerous hurdles for electric aviation to go mainstream, major shipping companies are clearly betting on a future where the technology plays at least some role in transporting goods.
Correction: An earlier version of this story indicated the Air Force has a purchase agreement with Beta rather than a partnership. This story was updated to reflect this difference on Sept. 8, 2022.

Lisa Martine Jenkins is a senior reporter at Protocol covering climate. Lisa previously wrote for Morning Consult, Chemical Watch and the Associated Press. Lisa is currently based in Brooklyn, and is originally from the Bay Area. Find her on Twitter ( @l_m_j_) or reach out via email (ljenkins@protocol.com).
If you thought the rise of remote work, independent contractors and contingent workers rose sharply during the pandemic, just wait until the next few months when you see a higher uptick in the on-demand talent economy.
Rising workload and pace, the stress of commuting and a taste of the flexible work-from-anywhere lifestyle have all contributed to what many are calling the Great Resignation, which is only just the beginning of the headwinds organizations are facing, says Tim Sanders, vice president of client strategy at Upwork, a marketplace that connects businesses with independent professionals and agencies around the globe.
“It began with front-line workers, but it’s not going to end there,” Sanders notes, “Recent data suggests that the biggest industries for quits are now software and professional services and on top of that, I predict that we’ll see more leaders and managers continuing to quit their jobs.”
As the economy leans toward a recession, and layoffs across dozens of tech firms make headlines, Sanders predicts companies will increasingly turn to on-demand talent. “These highly skilled independent contractors and professionals offer the speed, flexibility and agility companies are seeking right now. Leaders are becoming more empowered to fully embrace a hybrid workforce and shift away from rigid models.”

Leaning into headwinds: Driving growth amid uncertainty
A recent report from Upwork, The Adaptive Enterprise, underscores the importance of flexible on-demand talent during uncertain times. Sanders notes: “A growing number of organizations, including Upwork and customers like Microsoft, Airbnb and Nasdaq understand that on-demand talent enables companies to reduce risk, drive cost savings, and at the same time, protect their people from burnout. Flexible workforce models also allow businesses to respond to and recover faster from crises than more traditional models.”
Some crises come in the form of economic slowdowns, while others can take the shape of geopolitical conflicts that disrupt life and work as we know it. Mitigating risk — such as a pandemic wave striking a certain region housing the majority of a company’s staff — is one reason businesses turn to on-demand talent, but it’s certainly not the only one.
CEOs surveyed by Deloitte in 2022 see talent shortages as the biggest threat to their growth plans. The survey goes on to report that CEOs believe that talent is the top disruptor to their supply chain and there is more to be gained within their workforce by providing greater flexibility (83% in agreement) as opposed to merely offering more financial-related incentives. What is top of mind for many business leaders is needing to fill talent and skills gaps, so they can deliver new products and enhanced services. In other words, companies are struggling to find the specific skill sets needed to advance their business objectives and innovation agendas.
The biggest benefit of leveraging on-demand talent is often tapping into the talent and skills that businesses can’t find elsewhere. Upwork’s recent report highlights that 53% of on-demand talent provide skills that are in short supply for many companies, including IT, marketing, computer programming and business consulting.

By harnessing a global talent pool from digital marketplaces like Upwork, businesses have wider access to skilled talent who can accelerate what those companies offer to customers at a fraction of the cost. “Skillsourcing” on-demand talent helps companies maintain a more compact population of full-time employees to concentrate on work that only they can do as well as maximize their strengths while bringing in independent professionals to handle the rest.
Behind the growth: Speed, flexibility and agility
Speed, flexibility and agility are three critical benefits offered by on-demand talent to businesses seeking competitive advantages in their sector. While on-demand talent solutions give companies speed-to-market advantages, Sanders sees that they also give organizations a strategic form of flexibility.
“An agile organization is able to make bold and quick moves without breaking everything,” Sanders says, “and look at a number of our Fortune 100 customers that have a workforce made up of almost half on-demand talent, and how they can pivot on a dime. It’s a case of structure enabling strategy.”
As for speed and efficiency doing the actual work, Sanders says clients report that when hiring managers have been given access to on-demand talent, they engage the needed talent within days instead of months, and when they bring them onto projects, the work is completed up to 50% faster than through traditional avenues.
Sanders says, “Businesses have realized that remote work experiences are best led and judged by outcomes, not just time in the office, and more leaders are comfortable and confident opting for a hybrid workforce that can deliver based on those outcomes.”
Upwork’s Labor Market Trends and Insights page shows that organizations are indeed ramping up their hybrid workforces: 60% of businesses surveyed said they plan to use more on-demand talent in the next two years.
“The old way of acquiring talent isn’t efficient,” Sanders says. “Staffing firms aren’t the silver-bullet solution they once were, and more businesses need to rethink and redesign their workforce with on-demand talent as the economy and work rapidly evolve. The conversation is no longer about the future of work, but the future of winning.”

After long ignoring streaming services, political campaigns are finally acknowledging that media consumption habits have changed.
Will political ads come to Netflix this midterm?
Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety’s first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.
The week after Labor Day traditionally marks the beginning of the fall election season, with campaigns rushing to get their message on the airwaves before the November midterms. And this time around, cord cutters won’t be spared from the ad onslaught.
Campaigns from both sides of the aisle are expected to spend $1.2 billion on streaming ads this midterm season, according to recent estimates from analytics company Kantar. “Connected TV is really the big story in political advertising this year,” said Grace Briscoe, SVP of client development at Basis Technologies, a Los Angeles-based ad and marketing software company. “Every campaign we’re working with, up and down the ballot, even small state legislative races [and] mayoral races, they’re all looking to explore connected TV.”
That’s a welcome reprieve for the streaming industry, which has seen spending on ads slow down due to supply chain shortages, late-pandemic uncertainties and inflation woes. The surge in political ad spend is also giving political campaigns a chance to target audiences that have long eluded them, and it’s setting up these operations for a quickly approaching post-cable future.

“Anywhere from one in three millennials to as many as half of Gen Z adults report they no longer have linear television services,” said Samba TV SVP Dallas Lawrence, who used to work in Washington, including at the White House, before joining the ad tech and streaming industry. “These voters are 100% unreachable by traditional TV campaigns, and they are turning out to vote in ever-greater numbers.”
Not too long ago, political advertisers all but ignored streaming. “Political campaigns are traditionally pretty risk-averse,” Briscoe said. “They’re usually slow to adopt new things.” That’s largely due to the cyclical nature of a business that measures every new campaign against prior ones. “If you’re a political consultant, you do what you did last time, and the time before that, and the time before that,” she said. “You don’t mess with a formula that works.”
Until it doesn’t, that is. Ad spending is increasingly out of touch with how people actually watch TV, which became obvious during the 2020 election. “In the critical final month of one of the most expensive Senate races in the country, 90% of all political TV ads reached the same 55% of voters,” Lawrence said. By focusing solely on cable and broadcast TV, campaigns effectively ignored nearly half of the electorate. “For younger voters, [streaming] is really the only way to reach them,” Lawrence said.
Some campaigns have finally woken up to this new reality, with Basis Technologies seeing a 15x increase in the money spent on streaming ads during the first six months of this year, compared to the same time span in 2020. “We’re seeing this dramatic shift of connected TV spending this year,” she said.
Basis isn’t the only company noticing campaigns moving toward streaming in a big way. “This year, we’re seeing a very large swath of advertisers relying on [streaming],” said Mark Jablonowski, the CTO and managing partner of ad network DSPolitical. The company, which primarily works with Democratic and progressive campaigns, has seen connected TV ad budgets swell to 20% compared to mid-single-digit spending just two years ago. “It’s becoming a very big thing,” Jablonowski said.

One reason for this trend is the growth of ad-supported streaming. Back in 2018, Hulu and YouTube were the only two major players in the streaming space that sold ads, and Hulu long restricted political advertising.

Since then, Viacom has acquired Pluto TV, Fox has bought Tubi and NBC Universal has both bought Xumo and launched Peacock with an ad-supported tier. HBO Max launched its ad-supported tier a year ago.
Democrats in Hulu font Advertisers have been missing the cord cutter market.Image: Christopher T. Fong/Protocol
All the activity has led to a lot more inventory being available to political advertisers, which allows them to more precisely talk to constituents in individual markets. “This is the first year where we’re able to offer connected television individually targeted at scale,” Jablonowski said. “It’s a big switch in the industry.”
It’s a shift that isn’t lost on the major media companies. Paramount Global CEO Bob Bakish specifically called out Pluto as one of the services benefiting from this year’s midterm spending on his company’s Q2 earnings call; NBCUniversal CEO Jeff Shell also acknowledged that streaming will play a growing role in bringing in those political ad dollars. “We expect some pretty strong results from Peacock in the coming fall,” Shell recently told investors.
Roku CEO Anthony Wood echoed those remarks. “We expect political [advertising] to continue to grow, continue to be an important vertical for us,” Wood recently told investors. At the same time, Wood acknowledged that his company can’t fully benefit from the height of the campaign season just yet. “Political advertising tends to be in certain, very high-demand, localized markets,” he explained. “And so, even though we have a lot of scale in a particular market, we’ll reach caps fairly quickly.”
In other words: Roku and others simply don’t have enough ad spots to sell yet.
That could change in a matter of months: Toward the end of this year, Netflix plans to launch ad-supported plans for its respective streaming services. A Netflix spokesperson declined to comment on whether the company would allow political ads. Disney+, which also plans to launch an ad-supported tier later this year, will not accept political ads “at launch,” according to a Disney spokesperson.

Disney has been justifying its resistance to political advertising with the goal of keeping Disney+ family-friendly. However, the service has been evolving from its family-focused roots in recent months, including with the addition of R-rated fare. Disney also recently changed its policies around political ads on Hulu, and there’s nothing that would stop the company from instituting similar changes for Disney+.
“Should Netflix and Disney+ allow campaign spending, it will kick off a land grab unlike anything we have seen in more than a generation of political TV advertising,” said Lawrence. “The ability to combine targeting and measurement insights with the storytelling power of franchises like ‘Marvel’ and ‘Star Wars’ will be a game-changer for political campaigns.”
Until then, limited inventory is likely to drive up prices for streaming ads, which could have ripple effects on non-political advertisers. “I expect that we are going to see some surges in really competitive bidding in connected TV, particularly in states with competitive, high-profile races,” Briscoe said. “If you’re a retailer in Texas, it’s going to be tough.”
Briscoe expects that some companies will respond to this by temporarily pulling back on ads in certain markets, or just wait out campaign season altogether — unless they are forced to compete, no matter the price. “If you’re doing a movie release, you’ve got no choice … you’re going to pay more,” Briscoe said. Until everything returns to normal in a few weeks, that is.
“October is going to be very competitive,” Briscoe said. “But come Nov. 8, all those advertisers are done.”
Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety’s first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.
It’s not just collectibles. Fantasy sports and fan service are fueling growth in NFTs.
Sorare, which started with a fantasy NFT soccer game and added Major League Baseball this summer, announced an expansion into NFT gaming with the NBA Wednesday.
Tomio Geron ( @tomiogeron) is a San Francisco-based reporter covering fintech. He was previously a reporter and editor at The Wall Street Journal, covering venture capital and startups. Before that, he worked as a staff writer at Forbes, covering social media and venture capital, and also edited the Midas List of top tech investors. He has also worked at newspapers covering crime, courts, health and other topics. He can be reached at tgeron@protocol.com or tgeron@protonmail.com.
The crypto market has been rocky in recent months and some NFTs have seen their prices crater, but digital tokens for sports are still scoring points with fans and investors.
Several companies are working to bring mainstream sports into the world of NFTs, with the support of major blockchain protocols that are eager to show broader utility. Sports NFTs are now mostly used for collecting or playing fantasy sports, but many expect NFTs to also serve as ways for fans to connect with their favorite teams or players. The NFT sports collectible market alone could be worth $92 billion in a decade’s time, by one analysis.
Sales of Dapper Labs’ NBA Top Shot NFTs, which feature collectible video “moments” that can be bought, sold and traded, have continued to grow and the company has expanded to UFC and NFL collectibles. Most recently Ticketmaster announced a deal to use the Flow blockchain, which Dapper Labs created, to attach NFTs to tickets.

Meanwhile, Sorare, which started with a fantasy NFT soccer game and added Major League Baseball this summer, announced an expansion into NFT gaming with the NBA Wednesday. The basketball game, which is expected to launch early in the NBA season this year, will feature tournaments using NFT-based player cards like its other games, but with features specific to the NBA. It will also have a marketplace, like its other games, where people can buy and sell cards.
Since Sorare announced its MLB product in May it has seen 250,000 new signups and $5 million in MLB trading volume so far.
Sorare’s basketball fantasy NFT game enters a market that already has a popular NFT collectibles product in NBA Top Shot as well as traditional fantasy basketball games on sites like ESPN and Yahoo. There’s also betting through sites like FanDuel.
Michael Meltzer, Sorare’s head of business development, sees an evolving landscape where NFTs can be used for a variety of purposes from ticketing to games and doesn’t see direct competition from other NFT sports providers.
Over time, he said, the industry will view Sorare’s games as “completely a different category than what some other companies are doing,” Meltzer said.
Fans benefit when there are many different ways to interact with teams and leagues, said Jorge Urrutia del Pozo, vice president of football at Dapper Labs. (That’s football as in soccer.)
Dapper is preparing to roll out a bilingual NFT collectibles product with the Spanish soccer league LaLiga, which has large fan bases in Spain, South America, Indonesia and the Middle East. The product will have access to 15 years of archives for its products, including stars like Lionel Messi and Cristiano Ronaldo.
But for sports leagues, having more touch points with fans, from TV to game tickets to merchandise to fantasy games to NFTs, makes things more complicated. Above all, they’re trying to understand their fans.
Sorare's NBA product. Sorare has a multiyear deal to be the official NFT fantasy sports provider for the NBA.Image: Sorare
“Managing that and getting what they call the single view of the fan — Tomio likes going to Warriors games and bought a jersey for Steph Curry so I’m going to infer that this is his favorite player … managing that is a tremendous challenge,” said Urrutia del Pozo.

Partners need to feed data to them and they need technology to make sense of it all to figure out what to market to fans and what to sell to them. This is important if NFTs can be used as proof of attendance or to provide rewards, he added.
Meanwhile, the protocol Algorand is working with FIFA, which is creating its FIFA+ Collect NFT marketplace. It will have digital moments from FIFA World Cup games and other art and images.
Sean Ford, interim CEO at Algorand, sees sports as well as games and music as key drivers of growth for blockchains like Algorand. It’s also a way for leagues to engage and connect more with fans, he said.
“The reason I say sports, music and gaming as well is that underlying all three are very passionate and deeply connected user bases. If you look at that emotional connection of the fans, and consumers of that type of creative content, it is definitely at a level that is well beyond, say, what you’d see for your favorite video conferencing service,” Ford said.
But the opportunities for NFTs in sports go far beyond just collectibles or fantasy games, said Ford.
Sports teams or leagues can use NFTs to sell fractional shares of teams, to sell seat licenses, to grant fans access to special areas in the stadium or special merchandise or to allow them to get unique digital moments of a game they attended. It’s a way to increase engagement, which is, after all, the goal.
Tomio Geron ( @tomiogeron) is a San Francisco-based reporter covering fintech. He was previously a reporter and editor at The Wall Street Journal, covering venture capital and startups. Before that, he worked as a staff writer at Forbes, covering social media and venture capital, and also edited the Midas List of top tech investors. He has also worked at newspapers covering crime, courts, health and other topics. He can be reached at tgeron@protocol.com or tgeron@protonmail.com.
Edward Lee, a professor emeritus at Berkeley, wrote a blog post chastising the rejection culture at computer science conferences. Here’s the story of how he decided to write it and why it went viral.
“I have seen some extremely talented people leave the field because of brutal reviews. And that’s just unacceptable,” computer science professor Edward Lee told Protocol.
Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.
Click banner image for more How I decided series
Wannabe computer science superstars must all run the same rather scary and capricious gauntlet, one that sounds deceptively dull: the computer science conference paper review process. To have a research paper accepted for presentation at a CS conference is a coveted rite of passage among academics and professionals, bestowing on its author a status symbol that can open the door to tenure or competitive job offers.
Last month, the University of California, Berkeley’s much-respected Edward Lee, a professor emeritus of electrical engineering and computer sciences who for several decades has served on program committees that judge research papers, caused an uproar in the CS community after he publicly shared a scathing review of the system, which he’d sent earlier to fellow judges. Program committee members who decide which papers are accepted are volunteers, members of the academic community who agree to spend hours of their time (theoretically) reading submissions, writing opinions and voting on whether papers are worthy of the hallowed halls of whatever conference is in session.

But ever since conferences adopted a new review process that shields the names of judges, as well as papers’ authors — many made the change in the early 2000s — critics say a new problem has arisen: Rejection notes are often so random, or just factually incorrect, that applicants suspect nobody actually read their paper. The issue came to a head for Lee earlier this year. Here is his story of why he’s chastising the community he’s been a loyal member of for so long, and what he thinks can be done to address the problem.
Lee’s story, as told to Protocol, has been edited for clarity and brevity.
It started when I was serving on a program committee for one of my favorite conferences. It was that program committee experience that pushed me over the edge, and I wrote a letter to the entire program committee and resigned.
I had found myself fighting with a lot of the program committee members over determinations about papers, and a number of papers that I thought were highly worthy got rejected. There were two papers that were principally authored by students that I had worked with closely, and of course I couldn’t participate in the deliberations because of conflict of interest rules. But those papers got rejected with what I considered to be unsound reviews. And I have enough experience to know that these two papers were excellent papers.
My resignation and protest letter got quite a few people upset. I got quite a bit of feedback. It has become very clear to me that there are a lot of people who are very frustrated with the current situation. This piece seems to have really resonated with a lot of people because everyone in the community is facing 10% acceptance rates for their papers. I have seen some extremely talented people leave the field because of brutal reviews. And that’s just unacceptable.
Employers need to know the reality is that getting conference papers accepted is extremely random. Looking at published conference papers in computer science as a measure of the quality of the candidate is flawed. You’re looking at luck. If you want to hire lucky people, OK. That’s usually not what they’re looking for.

And then the Sigbed blog editors somehow got wind of my open letter to the program committee and asked me if I would submit it as a blog, which is how the Sigbed blog post came about.
I’ve been doing these kinds of reviews for my entire career. So that’s 40 years. I get invited to be on a lot of these program committees, but I simply don’t have the bandwidth for them. So I typically serve on two or three a year, trying to pick the type of conferences that I can contribute the most to.
The problem has been there all along, but it was much less visible to me because the reviews didn’t use to be double-blind. The students that I worked with the most closely were almost always from Berkeley, and Berkeley papers weren’t rejected as often as papers from other places.
So in some ways, the institution of double-blind review processes has been a very good thing, because there were prejudices creeping into the review process unknowingly. The papers from the best institutions were more likely to be accepted, papers written by males rather than females were more likely to be accepted. Papers with Chinese names were more likely to be rejected. The double-blind review process put an end to that problem.
But that also exposed to me the high rejection rates and the frustration that accompanies them, because the reviews are frankly capricious and often unsound.
Part of the problem is that the program committees are being asked to do more than is actually possible. In the past, they could rely on a kind of a crutch. It’s an MIT paper, it’s probably pretty good. Let’s just accept it. But they can’t do that anymore.
There’s also the anonymity. There are good reasons for keeping the reviewers anonymous — you don’t want junior people who are reviewing to be vulnerable to retribution from senior people who get their papers rejected. But people can be much more mean when they are anonymous. And moreover, when you combine that with the fact that reviews themselves never get published, their critique of that paper is protected.

One thing that we could do that would improve things quite a bit is keep the double-blind process, but the original submission and the reviews get published, right along with the paper. That way the conference gets associated with the reviews, and if the conference has a lot of capricious reviews, that’s going to degrade the reputation of the conference. Right now there’s basically a lot of power with no accountability, which is almost never a good thing.
The first open letter that I sent got circulated to all the new program committee members in another related conference just shortly thereafter. I’ve seen quite a bit of discussion about being a lot more careful about using novelty as a criteria for rejection, for example, which is one of the things I argue against in this blog.
I’m hoping that there will be some impact. I’ve been collecting notes from all the feedback I’ve been getting, and I might have enough to put together a more upbeat follow-up blog that discusses some real concrete actions that can be taken.

Wednesday, June 29
How I decided to exit my startup’s original business
Wednesday, July 6
How I decided to shape Microsoft’s climate agenda
Wednesday, July 13
How I decided to cap hiring at our high-growth software startup
Wednesday, July 20
How I decided to allow remote work forever at Atlassian
Wednesday, July 27
How I decided not to pivot
Wednesday, Aug. 3
How I decided to move my music tech startup to London
Wednesday, Aug. 10
How I decided to leave Alphabet and build a geothermal energy startup
Wednesday, Aug. 17
How I decided to step down as CEO
Wednesday, Aug. 24
How I decided to swap the Silicon Valley C-suite for West Virginia
Wednesday, Aug. 31
How I decided on the right AI auditor for my hiring tech company

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.
To give you the best possible experience, this site uses cookies. If you continue browsing. you accept our use of cookies. You can review our privacy policy to find out more about the cookies we use.

source

Leave a Replay

Sign up for our Newsletter

Click edit button to change this text. Lorem ipsum dolor sit amet, consectetur adipiscing elit