The driver is likely automatically installed on your phone to support that function, but is not on fedora. This seems like a driver issue for that particular headset.
I just want to know, how about the basic control like pause and volume control, are those work? If works, may be you want to try with same app that also available on Android that works with swiping to switch a song, for example Spotify.
I believe the signal sent from audio device to computer should use same protocols or whatever it called. The translation of swiping or push button on the audio device should be processed with it independent process to convert it to signals that then send to computer.