Francesca McGhie and Alex Stewart's tries in the first half gave the Scots the lead, but Japan was still within striking distance at halftime thanks to a goal from Seina Saito.
After Ayasa Otsuka's penalty put Japan within four points, Rhona Lloyd walked over in the corner and scored the game-winning try.
#scotland