Most of the budget GPS watches are designed to track training and not really as navigation aids. There are some really cheap ones (like the Aldi Crane GPS that was under $100 on sale - from memory it was around $70) .
If you don't intend on using the watch to log your bushwalks (leaving the tracking constantly on, so it records a breadcrumb trail of your walk), then the battery life can be quite good. Most of them require the watch to be on a specialised USB cable and cannot be worn when charged, and the cheaper ones won't record GPS while they are being charged. But the worst case if you are using it away from a power plug is that you can bring a portable USB battery pack, and charge it up that way. The internal batteries are fairly small, so even a 5000mAh external battery pack would last a few recharges.
I looked at the Aldi one a few years ago, and remember that even though the specs said 16 hours or so with tracking on, someone on another forums said theirs lasted 24 hours or so. But the battery life will diminish as the watch gets older, and the inbuilt lithium battery is usually not user replaceable. Compare this with an old Garmin handheld GPS I have had for almost 10 years. I can just replace the AA batteries it takes and it still works. If it was a built in rechargeble lithium battery, I'd say I would have to find a replacement for it now.
Some of the other issue with these cheaper GPS watches are not specifically made for hiking are:
* There maybe no input for GPS coordinates on the fly. So you have to preload the waypoints via PC. You can usually still mark points on the watch, but you might be limited in the way you can name those waypoints.
* The memory for number of waypoints log might be limited.
* There may be no way to change the datum on board, so it comes down to you to manually calculate the long/lat location into a format that you can check on your paper maps.
* There are no detailed mapping on the very budget ones. You might get a small representation of your location and saved waypoints.
Have a look at DCRainmaker's great blog
http://www.dcrainmaker.com/product-reviews He does good reviews on various GPS watches (with a training focus).
The more expensive GPS watches, like the various Garmins have more features that would be suitable for bushwalking/navigation. The upcoming Epix has a nice looking touch screen and ability to have full topo maps on board, but even the Fenix/Tactix/Aquatix models could get a basic map on screen. However, they are not exactly "budget" GPS watches.
If you want to do more than that in a wrist mounted GPS device, have a look at the older Garmin Foretrex series. They don't have the training aspects of the budget GPS watches, but may do what you want it to do. Runs from AAA batteries, but also does not have the ability to add on maps.
An alternative for you is to buy a basic trail GPS like the Etrex 10 which has been about $90 on sale. These run on disposable or rechargeable AA batteries, and a few people on the forums have them. A step up to something like the Etrex 20 adds the ability to add maps to the device. The Etrex is nice and light - my Etrex 20 weighs around 100grams, plus the batteries (this can vary depending on the type of batteries used - 2xNiMH AAs are about 50 grams; 2xLithium AAs are about 30grams)