Share custom field with everyone again after it's been restricted

I have a number of custom fields, most of which are shared with everyone (the default when they created). I have restricted a few fields to a smaller group.

I now want to make one of the fields that is currently restricted to a specific team available to everyone. I can't find anyway to do this.

'Everyone' isn't an option in the drop down sharing list. If I delete the current share group, it reverts to just me having access. If I try to delete me it tells me no one will have access!

So how, after restricting a field to a specific group, can I make that field available to everyone again?