Bicycles are known and understood by the people you're sharing spaces with; there are rules for where they're allowed and where they're not allowed and how they have to behave. The Japanese laws essentially allow ebikes that mimic bikes very closely to be used as bikes; otherwise you have to follow the rules for a scooter/moped (which is, after all, what you are).
I'll be riding my bike in a designated a bike lane (which is sometimes shared with pedestrians, dogs, etc.) with clearly marked speed limit signs of 15mph. That's plenty fast in general, but even more so in busy spots. And suddenly a near-silent e-bike will go flying past me (no "On your left," by the way) at speeds close to 30mph. Happens all of the time and is bafflingly reckless and arrogant.
The way folks ride class 3 e-bikes they really should be treated as mopeds: put them on the roads like other motorized vehicles and licensed and (hopefully) forced to obey the rules of the road. Just my opinion, but they seem to break the spirit of being in the bike lane. The touristy day-cruiser eBikes like Lime bikes are fine.
Legally, in many places they are mopeds. The problem is that there is no way to enforce the law because they look like e-bikes. It takes checking model numbers of bike or components. Throttle is relatively easy to notice.