The Serious Fraud Office (SFO) was forced to bring in the Information Commissioner's Office (ICO) after it managed to lose 32,000 document pages and 81 audio tapes and electronic media which formed part of a case.
The lost data was part of the bribery probe into BAE's al-Yamamah deal with Saudi Arabia. The investigation into the huge arms deal was discontinued in 2006, after intervention from then-prime minister Tony Blair.
The SFO said it lost the items when it returned more material than intended, to a source in the investigation.
An SFO spokesperson said: "The SFO is dealing with an incident of accidental data loss. The data concerned was obtained by the SFO in the course of its closed investigation into BAE Systems.
"The SFO has a duty to return material to those who supplied it, upon request, after the close of an investigation. In this instance the party requesting the return was sent additional material which had in fact been obtained from other sources."
The large amount of lost data in question constituted just 3 percent of the total data in the mammoth case. Of the lost data, said the SFO, 98 per cent of the material had been recovered and "efforts continue to recover all the remaining material that has not already been destroyed by the recipient".
It says no material relating to national security was included in the data.
Following the intervention of the ICO, the SFO brought in Peter Mason, former director of security at the Palace of Westminster, to conduct "an initial review" of the incident. He has since made some recommendations.
These include the continuing ownership of data in a concluded case by designated operational staff, a re-drafting of the responsibilities of the SFO's senior information risk owner, and raising the profile of data handling as a "key risk" in the SFO's business
The SFO has also instigated an "independent wide-ranging review" of all the organisation's business processes by Alan Woods, a former senior civil servant.
The data loss took place between May and October 2012 but was only discovered in May this year, said the SFO. The SFO said it had contacted 59 suppliers of data for the investigation to inform them of the situation.
The al-Yamamah deal involved the sale of tens of billions of pounds worth of arms by BAE Systems to Saudi Arabia, which began in the 1980s and which ended in 2006 with the sale of Typhoon fighter jets.
Allegations of corruption and bribery led to an SFO investigation in 2004, but the probe was closed in 2006 on the grounds of "public interest", amid political concerns that relations with Saudi Arabia were being harmed.
BAE paid $450 million (£300 million) in fines in the UK and US three years ago, to end other corruption investigations involving deals in Saudi Arabia and Tanzania.
Labour MP Emily Thornberry, shadow attorney general, said the data loss incident "raises more questions than it answers", and questioned what the government was doing to "ensure that this never happens again".