"We will never yield to any provocative act. We will maximise our pressure," on Pyongyang, Shinzo Abe told reporters, adding that the missile was thought to be an ICBM.
Japan's defence minister said earlier the missile was thought to have landed within Japan's exclusive economic zone (EEZ).
The Pentagon said the missile flew about 1,000 kilometres before splashing down in the Sea of Japan.
Japan had "completely tracked" the missile, Abe told reporters.
"We have made a strong complaint," he added.
By launching the missile, North Korea has ignored the international community's "united, strong will toward a peaceful solution," said the prime minister.
"The international community needs to perfectly implement sanctions, in unison.