In my experience, taxis are a bit hit-and-miss. Sometimes they will take you to Victoria Falls lookout, but on occasion they have wanted to charge me a hefty car-cleaning fee if they leave the bitumen. As wildwanderer said, the fire trail is scenic, being largely on the ridgetop. I get taxis from the rank at Blackheath (main traffic lights) since they are plentiful there.
One option at the Blackheath end is to exit the valley at Govett's Leap (which is a spectacular exit track). A couple of hundred metres up the road is the NPWS Visitor Centre, where there is a bus to Blackheath and Katoomba (approx hourly mon-fri, less frequent on weekends,
PDF timetable is here). I don't know whether it accepts cash fares but it definitely takes Opal Card, which you would need to buy in advance.
Maybe I've just been lucky atop Solitary, but in Spring I've always found running water in Singajingawell Creek (though not always by the campsite at the head of the stream; sometimes downstream 10 minutes). I've also always found water in the tank below Ruined Castle. Plenty of firewood at Singajingawell too, once you are 50 metres from the campsite.
Exiting to Wentworth Falls is probably the easiest option, but it involves a long suburban road walk after you leave the valley. Everyone has their own preferred circuit of Mt Solitary but here's my favourite:
Walk from Katoomba Station out along Glenraphael Drive (the fire trail on Narrowneck) then descend the Golden Stairs to the Federal Pass track. (Alternatively, walk or take the bus from Katoomba Station to Scenic world, then walk down Furber Steps or ride the Scenic Railway, then take the Federal Pass track (avoiding the boardwalk, or doing the boardwalk as a side-trip) past the landslide to the bottom of the Golden Stairs.) Continue to pass below Ruined Castle (or drop your pack and visit the summit on a side-trip). Collect water from the tank if you wish. Climb the ridge (there are a few straightforward scrambles, but if it gets hairy you're off the route). Check out the caves in Chinamans Gully, and walk in both directions to the viewpoints (100 metres or so, faint track, probable water near the northerly viewpoint which would need treatment). Continue across most of the mountain to Singajingawell Creek.
The next day, head down the steep eastern end of the mountain. At the junction part-way down, I prefer to keep going, cross the Kedumba River, pick up the track on the other side and climb to the fire trail where I turn left. There is no vehicular traffic on this fire trail. There are a few ups and downs, then when you are close to Katoomba there is a short link path on the left (signed) leading to Federal Pass track. Turn left, then right up Dardanelles Pass to the Giant Stairway which takes you up to the Three Sisters and then on to Katoomba Station (or follow the track to Echo Point for refreshments, and buses to the station). Alternatively, at the fire trail after climbing out of the Kedumba River, turn right and exit the valley on Kedumba Valley Road which leads to Wentworth Falls.
For a three day trip, camp the first night at Singajingawell and head down the eastern side of the mountain. At the junction, turn right and follow the signed route to the Kedumba Valley campsite (a short day). The next day head out on the Kedumba Valley Road. Most people doing a 3-day would camp the first night near Ruined Castle, but I prefer to camp high.
All of the route is mapped on
OpenStreetMap, so if you are using a phone app such as OruxMaps the navigation is easy if you download the map first. No phone coverage in most of the valleys, but I get coverage along the northern edge of Mt Solitary.
If you end up doing the Grose Valley from Mt Victoria to Blackheath, then the Mt Solitary circuit, I reckon you will have had a superb introduction to the upper Blue Mountains. If you have the odd spare day, some good day walks include the Grand Canyon near Blackheath, and from Glenbrook to Lapstone stations rock-hopping through Glenbrook Gorge.