Yes — oranges and other citrus are safe and vitamin-rich, but their acidity can cause digestive upset in quantity. A segment or two at a time is the right amount.
What it means for budgies
There's nothing specific to a budgie that changes the verdict on oranges — it's treated the same as for parrots generally. A budgie portion is tiny — a single bite or a little grated amount is plenty.
The full picture on oranges
Citrus is packed with vitamin C and most parrots enjoy it, but the citric acid that makes it refreshing can irritate a bird's digestive tract when overfed, and very acidic diets are suspected of aggravating iron absorption issues in some species.
Peel and offer one or two segments at a time.
Remove seeds for small birds — not toxic like apple seeds, but a choking hazard.
Skip orange juice: it's the acid and sugar without the fiber.