Manage Members
Open a Segment to see Users in segment, and use Manage Members to add or remove contacts from it by hand.

Note
If the Segment is populated from a connected provider (Customer.io), membership is driven by that connection instead — manual changes here may be overwritten on the next sync.