Yes — parrots can't feel capsaicin heat at all, and chilies are rich in vitamins A and C. A whole fresh chili is a favorite treat for many species.
What it means for budgies
There's nothing specific to a budgie that changes the verdict on chili peppers — it's treated the same as for parrots generally. A budgie portion is tiny — a single bite or a little grated amount is plenty.
The full picture on chili peppers
Birds lack the receptor that makes capsaicin feel hot — biologists believe chilies evolved this way precisely so birds would eat them and spread the seeds. A habanero that would ruin your afternoon is, to a macaw, just a crunchy red fruit packed with vitamins A and C. Bell peppers, seeds and membranes included, are equally safe.
Offer whole fresh chilies or bell pepper chunks, seeds and all.
Wash well; organic when possible.
Dried chilies (unsalted, unsmoked) make great dry-bowl additions.