Seems to me that everything but the spider would move its full distance minus the width of the wall. The spider would crawl up and over the wall and would move its full distance minus four inches and the width of the wall.
It seems to have always been the consensus of the people I have been playing with, and this is going back to the first introduction of the rule in the late 2000s, that spiders treated all terrain (obstacles, walls, trees) like open ground that it could walk on but that it had to pay the movement cost to do so just like a real spider. The swift movement rule's first sentence is a statement about what the spider can do (climb on all terrain) and then the second sentence tells us the applications that are a result of that power. The implication my group has taken from that is that the ignoring is linked to the climbing. In that sense, we read "ignoring obstacle" as meaning that it ignores the rules for obstacles and treats all terrain the same as a byproduct of being able to climb on any surface in the same way as a real spider. This is also why we never make it take climb tests for anything, even though the rule doesn’t state the model with the rule doesn’t have to take such tests; only that the model is able to climb on any surface. Strictly speaking, that wording is an exception to unscalable terrain rule on pg. 30 and not to the rest of the climbing section on the same page. Even though the swift movement rule also says that difficult terrain is not treated as having any slowing effect.
To me, the climbing on the surface is what makes it distinguishable from say, an ent, stepping over the obstacle or a man vaulting it because these methods don’t make the model take any extra steps. Meaning, no additional movement is absorbed by the end or the vaulting man.
Now, is there an argument to be made that the true purpose of the swift movement is to allow the spiders to treat obstacles as open ground and to basically phase through them? Sure. I could see reading the rule with that interpretation given above wasn’t already long standing, and well accepted precedent. I think the rules lay out what an obstacle is well enough that no one would disagree with a spider being able to ignore a fallen long but not be able to phase through a building. That said, I think Asamu’s position on the obstacles vs general terrain determination being model specific and based on the height of the model in question is sound given how the rules for obstacles, jumping, climbing, and barricades are all written to use the height determination. (See Asamu at
https://www.reddit.com/r/MiddleEarthMiniatures/comments/ujrned/swift_movement_rule/). Therefore, I’d say that the spider in your hypothetical would need to spend 2 inches of movement to climb the wall and then one inch to climb part way down the wall before falling the rest of the way without taking damage. So, the spider would end up being its full move minus three inches and the width of the wall away from the wall at the end.
This of course, suppose that the logic about the spider not needing a climb test from my earlier explanation holds up. Technically the rule only says it can climb on any surface. There is no mention of it not needing to take a climb test when it wants to climb something.