The grounds at Arlington Court are largely informal, with large clumps of rhododendron, hydrangeas, specimen trees and drifts of wildflowers in the spring. The exception is the one acre Victorian garden, which extends across three levels of grass with a conservatory as its centrepiece. The herbaceous borders flanking conservatory are full of colour throughout the summer from perennials while the high wall behind hides the newly restored kitchen garden. Circular flower beds have been cut into one of the terraces to recreate basket beds filled with bedding plants, and enclosed by ironwork frames through which cobaeas grow. The looped pattern is echoed by the arches of honeysuckle which flank a goldfish pond and fountain on the middle terrace of the garden. To the south of the Victorian garden are ferns, pampas grass and gunnera.

The newest jewel in Arlington's crown is the walled kitchen garden which has been restored to full working order with heritage varieties of fruit and vegetables growing. The produce is used in dishes in the tearoom and sold as veg boxes from the shop.