Rimi, 46, a deputy to al-Qaeda leader Ayman al-Zawahiri, was placed on the United States' most-wanted terrorist list after taking over al-Qaeda's Yemen affiliate in 2015.
US government had offered a USD 10 million reward for information on Rimi.
"His death further degrades al-Qaeda in the Arabian Peninsula (AQAP) and the global al-Qaeda movement, and it brings us closer to eliminating the threats these groups pose to our national security," Trump said on Thursday.
He said the major counter-terrorism operation was carried at his direction.
"Rimi joined al-Qaeda in the 1990s, working in Afghanistan for Osama bin Laden. Under Rimi, AQAP committed unconscionable violence against civilians in Yemen and sought to conduct and inspire numerous attacks against the United States and our forces," Trump said.
"The United States, our interests, and our allies are safer as a result of his death. We will continue to protect the American people by tracking down and eliminating terrorists who seek to do us harm," he said.
While Trump confirmed that Rimi had been killed, he did not say when the US operation was conducted or divulged any details about how it was carried out.
The AQAP has long been considered al-Qaeda's most dangerous branch for its attempts to carry out attacks on the US mainland.
Rimi had said in an 18-minute video that his group was responsible for the December 6 shooting at US Naval Air Station Pensacola in Florida, in which a Saudi Air Force officer killed three American sailors.
He called the shooter Mohammed Alshamrani a "courageous knight" and a "hero", according to media reports.
The shooting focused public attention on the presence of foreign students in American military training programs and exposed shortcomings in the screening of cadets.
In January, the US sent home 21 Saudi military students, saying the trainees had jihadist or anti-American sentiments on social media pages or had "contact" with child abuse images, including in internet chatrooms.
Rimi was considered a potential successor to al-Zawahiri, the Egyptian leader of al-Qaeda's strategic operations who is reportedly believed to be in Pakistan.
This is the third major strike by the US under Trump administration and that too in recent months. In October, the US killed ISIS leader Abu Bakr al-Baghdadi. In January, it killed top Iranian General Qassem Soleimani.