We parked in the castle car park (it's £2 but it gives you £2 off the cafe and as we always need to stop for some cake this was a bargain!) and walked past the castle, over the stone bridge and towards the woodland. There are clear blue signs from this point that lead you up to the monument. This isn't a long walk and is only slightly steep so no specialist training needed for this one! Once at the top there are gorgeous views of Inveraray, Loch Shira and the surrounding hills.
If you're visiting the castle (or even if you're not) I recommend this as a nice afternoon stroll with some impressive views.