Hours after Mattis landed, the rockets were fired at Kabul's Hamid Karzai International Airport from an unknown location and landed in an open area, according to Najib Danish, spokesman for the Afghan Interior Ministry. He did not reveal how many rockets were fired.
Mattis and NATO Secretary General Jens Stoltenberg had already left the airport at the time of the incident, Danish said. No one was injured.
The Taliban has claimed responsibility and said Mattis' plane was the target of the attack, the Taliban's spokesman, Zabiullah Mujahid, tweeted.
Up to 40 rounds of munitions hit the airport, according to a US military official, 29 of which were rocket-propelled grenades. The nature of the other munitions was not immediately known.
The official said the munitions were fired toward a guard tower and hit the south side of the airport.
An Afghan special forces unit was searching houses close to the airport on the suspicion that the rockets were fired from them, Danish added.
Mattis' unannounced visit Wednesday was his first trip to the country since US President Donald Trump announced a new Afghan strategy
in late August.
At a press conference with Afghan President Ashraf Ghani and Stoltenberg, Mattis said he had only heard some press reports but said "an attack on an airport anywhere in the world is a criminal act by terrorist."
"If, in fact, this is what they have done, they will find the Afghan Security Forces continuing on the offensive against them in every district of the country right now," Mattis added.
Mattis met with Afghan President Ghani at the Presidential Palace, alongside Stoltenberg and Gen. John Nicholson, commander of US forces in Afghanistan.
Saying it was "refreshing" to be in the country alongside NATO's leader, he said a "lot was riding" on Trump's new strategy.
"A lot is riding on this of course as we look toward how do we put an end to this fighting and the threat of terrorism to the Afghan people, to the international community and how do we put this into a path of political reconciliation," he said.
At the time, Trump said he was shifting the policy to take the emphasis off troop numbers, and would rely on greater regional cooperation to stabilize the security situation in Afghanistan.
"We are here to set the military and security conditions for that but recognize ultimately the responsibility for the Afghan leadership under your direction," Mattis added.
Earlier Mattis met with key Afghan government officials, including Defense Minister Tariq Shah Bahrami and Interior Minister Wais Barmak, as well as Stoltenberg and Nicholson, at Resolute Support Headquarters.
Nicholson gave an assessment of the fighting season so far, saying it had gone as well as can be expected.
Stoltenberg also spoke at the meeting, reiterating NATO support for Afghan forces.
"We are ready to train, assist and advise. We would like to help you break the stalemate that we have seen," he said.
During his visit, Mattis is also expected to meet with the and spend time with US troops.
Later, he'll hold a press conference with Ghani and the NATO chief.
Last week, Ghani used a substantial part of his speech at the United Nations General Assembly to praise Trump's recently announced strategy for Afghanistan and South Asia.
"With President Trump's recent announcement of his strategy to counter terror and stabilize South Asia, Afghanistan's enduring partnership with the United States and the international community has been renewed and redirected," Ghani told the audience at the UN headquarters in New York.
In his August prime-time address, Trump outlined his plan for the war in Afghanistan, vowing that the US would "fight to win" while no longer "nation-building."
The President declared he would no longer announce troop levels but would focus on allowing US forces to target the Taliban and other terrorist groups wherever they were in Afghanistan.
"America's enemies must never know our plans, or believe they can wait us out. I will not say when we are going to attack, but attack we will," said Trump, before a crowd of US troops.
There are approximately 11,000 US troops in Afghanistan,
according to revised figures given by the Pentagon in late August. The number given was around 2,600 more than had been officially disclosed due to a change in its accounting procedures.
Before entering office, Trump had campaigned for a complete withdrawal of US troops from the country. The 16-year war has claimed the lives of more than 2,200 US troops, as well as thousands of allied forces. Tens of thousands of Afghan civilians and military have also died in the conflict.
Trump indicated in August that he'd changed his mind and said withdrawing from Afghanistan would dishonor those killed.
Crackdown on Pakistan's 'harboring of terrorists'
Trump's new Afghanistan plan also emphasized that it was a part of a broader South Asia strategy that relied on greater cooperation from neighboring India and Pakistan.
He had particularly harsh words for Pakistan, saying Washington could "no longer be silent about Pakistan's safe havens for terrorist organizations."
"The Pakistani people have suffered greatly from terrorism and extremism," Trump said. "Pakistan has also sheltered those same organizations that try every single day to kill our people."
Afghan and US officials have long accused Pakistan of taking insufficient military action against Taliban leadership internally.
Speaking at the UNGA last week, Pakistan Prime Minister Shahid Khaqan Abbasi rejected accusations the country hadn't done enough to combat terrorism.
"Having suffered and sacrificed so much due to our role in the global counter terrorism campaign it is especially galling for Pakistan to be blamed for the military or political stalemate in Afghanistan. We are not prepared to be anyone's scapegoat," he said.
Abbassi added that Taliban "safe havens" were located not in Pakistan but in "large tracts of territory controlled by the Taliban in Afghanistan."
Trump's Afghan strategy also calls for more help from India, which has long maintained strained relations with Pakistan. "India makes billions of dollars in trade from the United States, and we want them to help us more with Afghanistan," Trump said.
Mattis' visit to Kabul comes straight off the back of a visit to India. After meeting Indian Prime Minister Narendra Modi, the two released a statement saying "both sides had reaffirmed their resolve to further develop their strong strategic partnership."
The statement said they'd discussed greater cooperation on regional and global scale in pursuing their shared goals of "peace, stability and combating terrorism."