Ok, I resolved it. I downloaded pa_stable_v190600_20161030.tgz from PortAudio - an Open-Source Cross-Platform Audio API and make install it but that was messing up things. So I make uninstall it. Then used the standard Fedora dnf install portaudio and now the problem is resolved:
SoundWire Server v3.0
Initializing server network communication
Using default port number 59010
Initializing PortAudio
PortAudio reported default input device sample rate is 48000 (may not be accurate)
Audio stream info: sample rate = 48000, input latency = 0.0213333 sec
Audio input device opened successfully
SoundWireServer: Audio capture running
Well, now I’ve made SoundWire work using the wifi plus a usb connection and setting the ip of the usb tethering.