It seems obvious that a line of length 4 is longer than a line of length 2; but couldn't we just as easily say that the two lines are equally made up of an infinite number of points?
You are right that the points in a 4 inch line segment can be put into one-to-one correspondence with the points in a 2 inch line segment. Think of a line swinging through both line segments, the way a door swings through a shorter path nearer its hinge and a longer path further from the hinge. The swinging line matches any point in one with a point in the other. Therefore, they have the same number of points--an infinite number. However, that is not a strike against the claim that the line segments have different lengths. The points are dimensionless, and the length of a line segment is not a function of the number of its dimensionless points. So the 4 inch line segment is still twice the length of the other.