It only takes a few taps to remove someone from your friends list.
How to remove a friend
Open their profile
Tap the three dots in the top-left corner
Select Remove Friend
🎯 Just want some space? You can also use ghost mode to pause sharing your live location without removing them.