CPARS is slow
[Programming Note: I’m back from vacation and, yesterday, I shared more about what I’m doing when I am not writing GovContrActually. Namely, my colleagues and I have started TandemGov, a company that helps organizations navigate government contracting while leveraging their strengths. If you are interested in more fresh #govcon content or just wanna throw some support around, please follow our company page. Or send referrals and checks, I guess. But definitely follow. Thanks!]
A few months ago, we talked about the Contractor Performance Assessment Reporting System (CPARS), which the government uses to evaluate the past performance of vendors, and one significant way that CPARS isn't like Yelp[1]:
According to publicly available CPARS metrics, something like only 1-2% of all ratings receive a marginal rating! If there are two marginal ratings on a contract, instinctively you might conclude that the contract warrants a closer look.
After all, in Yelp, a 3.5-star rating is pretty boring! Sure, a 5-star rating is notable, but the juicy dirt is usually in the 1-2 stars reviews. Can you blame the government for pulling a negative report and giving it some extra weight?
Yes! Yes you can blame the government for pulling a negative report and giving it some extra weight. Because, unlike Yelp, CPARS doesn't give vendors an overall rating, it just gives contracting officers access to the individual ratings. Which means that contracting officers' selections are super important.
In other words, source-selection decisions are disproportionately influenced by individual low ratings (1) because there aren't many low ratings in the system, and (2) because the system doesn't provide aggregate ratings.
If an acquisition professional sees a "marginal" rating in a CPARS report, they're going to write that up as a weakness in the vendor's past-performance. A problem for vendors is that the government can selectively decide when to use a CPARS report, including a negative one. A problem for the government is that a negative CPARS report still needs to be relevant to the current requirement:
You can't tally up ratings from somewhat relevant contracts as if they were relevant contracts... If you're the government, then, this case stands for the proposition that choosing a "close enough" CPARS evaluation can create protest risk. Unless the rating is on point, you may want to think twice.
A recent GAO decision illustrates another problem for CPARS: it's slow.
In The Whiting-Turner Contracting Company, the vendor argued that the Navy "cherry picked" a negative report from CPARS, which had the effect of knocking the vendor out of contention for a multiple award IDIQ based on past performance. Predictably, the Navy argued that it was just fine to look at the negative report. The Navy argued that "while the protester’s past performance was generally positive, it was not uniformly exceptional and did not merit a rating of substantial confidence." Also predictably, GAO accepted the Navy's argument. Sad trombone for Whiting-Turner.
Notably, however, the CPARS report that the Navy relied upon wasn't even a final report! It was marked "pending" in the system when the Navy looked at it. Here's the GAO:
The agency assigned Whiting-Turner’s proposal a weakness considering the three ratings of marginal Whiting-Turner received for its performance on the WEAC project and because the assessing official stated that she would not recommend Whiting-Turner for future work.... The agency concluded its evaluation of Whiting-Turner’s past performance on August 30. On September 21, without notice to the agency, the assessing official modified the WEAC CPARS evaluation to reduce the number of marginal ratings from three to two and to change the overall recommendation to a positive one for Whiting-Turner. On October 4, the reviewing official for the WEAC CPARS evaluation approved the modifications.
Let's replay that timeline: the Navy reviewed the CPARS evaluation in August. By by October, that evaluation was changed. In August—when the Navy was reviewing the CPARS—Whiting-Turner had three marginal ratings and a "no recommendation." In October, Whiting-Turner had two marginal ratings and a positive recommendation! You can imagine that if the CPARS evaluation had been "correct" in August, perhaps there's a different outcome for Whiting-Turner.[2]
Whiting-Turner, for obvious reasons, argued that the Navy shouldn't have used a pending report, or at least should have updated the past-performance rating when the CPARS report was updated. But the GAO explained that an agency doesn't actually need to wait for finality for CPARS reports:
While the original WEAC CPARS was marked "Contractor Comment Pending Government Review," it did not contain or detail any of Whiting-Turner’s comments. Whiting-Turner has not provided us with any statutory or regulatory authority stating that the agency was required to check whether the WEAC CPARS evaluation had been updated solely because it was marked "pending." We also are not persuaded by Whiting-Turner’s argument that the agency should have somehow known that its comments pending review challenged the ratings, when the agency had no way of knowing the content of Whiting-Turner's comments. Therefore, as the evaluation of Whiting-Turner's past performance was completed prior to the update, we find nothing objectionable in the agency's failure to consider the more recent evaluation that was not available to it at the time it performed the past performance evaluation.
Tough break.
There's maybe a bit of realpolitik behind the GAO's decision, though. After all, the empirical data suggests that getting CPARS ratings from agencies is slow. Like, really slow. For example, according to a recent Inspector General report, the Department of Transportation "on average, missed [CPARS] deadlines by more than 12 months." Similarly, another recent Inspector General report found that—on a nonstatistical sample of housing contracts— the Department of Housing and Urban Development "did not consistently complete or submit evaluations in a timely manner in CPARS for any of the six contracts reviewed...The submissions to the system averaged from 3 to 12.5 months late."
Given the typical 3-year window that agencies look to for CPARS reports, a delay of one year means that the CPARS report is only useful for 55% of that 3 years.[3]
In other words, getting a CPARS rating is slow.[4] Can you blame the government for pulling a negative report, even if that negative report isn't actually a final report? You might want the answer to be "yes, you can blame the government." But, apparently, the correct answer is "meh, whatevs, it's fine." Them's the breaks.
The bad news is that if you do have a negative CPARS rating hanging out there—even in a pending state—it's going to stand out like an elephant in a tree. But the good news is that, by the time a negative report is published, it might be delayed enough and old and distant enough to not be relevant for long.
A win win, I suppose?
[1] Sometimes, I run drafts of these posts through LLMs just for kicks, and most of the time I don't get much back that's too useful. But this time, courtesy of Mistral, one particular result was too good not to share: "CPARS is like Yelp for government vendors, but with more red tape and fewer taco emojis." How great is that?!
[2] Or not.
[3] A CPARS report is due to be completed 120 days (4 months) after the period of performance. A 12-month delay means that the CPARS report is only available for 20 months (36 months - 12 months - 4 months = 20 months).
[4] If you are curious as to why CPARS reports are so slow, the Transportation IG report gets to the heart of it: "Various factors can contribute to an [office's] inability to register contracts within 30 days. When we asked why the [offices] were not meeting the 30-day requirement, DOT officials cited staffing issues—unfilled vacancies, attrition, and turnover of contracting personnel—as well as competing priorities due to a shift in agency workload, mission requirements, and collateral duties. For example, many [offices] indicated that workload priorities do not always include their CPARS duties, which impacts the [their] ability to perform their assigned CPARS roles."
This will surely ring true for many acquisition professionals. Still, that's not the whole story, because the report goes on to say: "Based on our review, National Highway Traffic Safety Administration (NHTSA) was relatively successful in meeting the 30-day registration requirement... NHTSA used an automated tool that reminds [individuals] to check CPARS for contracts available for registration to improve CPARS registration compliance. The system is a best practice that allows NHTSA to ensure contracts are usually registered in CPARS in a timely manner."
Yes, sure. It's a good practice to have registration reminders. Hear me out, though. Instead of an automated reminder for an overworked person to register a contract in a system, maybe the system could just automate the actual registration? Nah, that would be ridiculous...