This is probably due to your phone automatically going into sleep mode after a few minutes. The call must then reconnect each time. You can prevent this by touching your screen every few minutes, or solve it by changing your phone's settings:
- Iphone: 'Settings' > 'Display and brightness' > Automatic lock should be on never.
- Android: 'Settings' > Display > Sleep should be set to never.