You might want to use the connect command to connect to the modem port. For example to connect to port 1, from thePortServer TS prompt:
#> connect 1
atd.....(modem commands)
<Ctrl> [ .
This will use Control left-square bracket, period to disconnect from the port and return back to the PortServer TS prompt. Then you can exit and close your telnet session.