It seems to me that a good man-hour, not man-day, would be a good unit for Story Point.
This phrase sounds really, really strange, and not true. Where did you read that there is a correlation between the points of history and the ideal man-day? The ideal person days may have been used in the early days of Scrum, but for me, Story Points (SPs) are another matter ...
Story Points is a way to quantify the relative effort associated with a particular product backlog (PBI), which consists of several tasks. Some teams use the numerical size (for example, a scale from 1 to 10) to estimate the “size” of the PBI, others use the sizes of t-shirts (XS, S, M, L, XL, XXL, XXXL), some use the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, etc.). And by the way, did you notice that SPs don't have units?
If I used days, most of my problems would have ratings of 1/2 or 1.
So what? This would mean that you have small PBIs, which is nice (at least not for the most important). But do not forget that theoretically there are two levels of assessment in Scrum: the level of lag of a product in points and the level of lag of Sprint in hours. As I mentioned in the previous paragraph, the PBI consists of tasks, and they should be divided into tasks in the second part of the Sprint planning meeting. Then the tasks are evaluated in hours, and the 16h rule applies: the task should not exceed 16 hours. If so, it is too large and should be divided into smaller tasks (because we value large things too poorly).
Do you have any ideas why the use of ideal person days is most often mentioned in Scrum literature?
In any case, it is out of date. The situation may change in the future, but the current consensus should be measured in units without loss. Points are completely separate from any unit of time, and this is to intentionally avoid any comparison with a unit of the real world, labor productivity should be measured using speed (the number of points that a team can achieve in one iteration).