Socket disconnects immediately after sending setDesktopSize

[anderoonies]

Description
Immediately after calling sock.flush in setDesktopSize, the socket connection closes immediately, before receiving a response from the server.

Steps to reproduce the behavior:
Calling setDesktopSize with a valid width, height, sock, and flags

Expected behavior
Resizing the socket, or disconnecting due to a message from the server.

Screenshots
The last message before disconnect:

Client (please complete the following information):
MacOS 10.12, Chromium 81.0.4044.138 (Official Build) (64-bit)

Server (please complete the following information):

• VNC server: x11vnc
• WebSocket proxy: websockify

Additional context
The data from the proxy connected to websockify:

[anderoonies]

update: i also sometimes see One or more reserved bits are on: reserved1 = 1, reserved2 = 0, reserved3 = 0

[CendioOssman]

What does the log look like in the browser, from websockify and from x11vnc?

[anderoonies]

Websockify:

The VNC desktop is:      197d6a23c62f:0
PORT=5900
WebSocket server settings:
  - Listen on :1337
  - No SSL/TLS support (no cert file)
  - proxying from :1337 to :5900
127.0.0.1 - - [22/May/2020 14:12:19] 127.0.0.1: Plain non-SSL (ws://) WebSocket connection
127.0.0.1 - - [22/May/2020 14:12:19] 127.0.0.1: Path: '/user/7182/graphics/'
127.0.0.1 - - [22/May/2020 14:12:19] connecting to: :5900
handler exception: [Errno 107] Socket not connected

Browser:

Data URI scheme cursor supported
noVNC.js:7526 >> RFB.constructor
noVNC.js:7526 >> Display.constructor
noVNC.js:7526 User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36
noVNC.js:7526 Setting viewport to full display region
noVNC.js:7526 << Display.constructor
noVNC.js:7526 Setting viewport to full display region
noVNC.js:7526 << RFB.constructor
noVNC.js:7526 New state 'connecting', was ''.
noVNC.js:7526 >> RFB.connect
noVNC.js:7526 connecting to ws://localhost:443/user/1723/graphics/
noVNC.js:7526 << RFB.connect
noVNC.js:7526 >> WebSock.onopen
noVNC.js:7526 Server choose sub-protocol: binary
noVNC.js:7526 Starting VNC handshake
noVNC.js:7526 << WebSock.onopen
noVNC.js:7526 Server ProtocolVersion: 003.008
noVNC.js:7526 Sent ProtocolVersion: 003.008
noVNC.js:7526 Server security types: 1
noVNC.js:7526 Authenticating using scheme: 1
noVNC.js:7526 Authentication OK
noVNC.js:7526 Screen: 1280x12288, bpp: 8, depth: 8, big_endian: 0, true_color: 0, red_max: 0, green_max: 0, blue_max: 0, red_shift: 0, green_shift: 0, blue_shift: 0
noVNC.js:6195 Server native red-shift is not 16
_negotiate_server_init @ noVNC.js:6195
_init_msg @ noVNC.js:6266
_handle_message @ noVNC.js:5761
(anonymous) @ noVNC.js:5353
_recv_message @ noVNC.js:9814
noVNC.js:7526 Setting viewport to full display region
noVNC.js:7526 New state 'connected', was 'connecting'.
noVNC.js:7526 >> Keyboard.allKeysUp
noVNC.js:7526 << Keyboard.allKeysUp
noVNC.js:7526 Requested new desktop size: 479x78.1875
noVNC.js:7526 >> WebSock.onclose
noVNC.js:7526 WebSocket on-close event
noVNC.js:7526 New state 'disconnecting', was 'connected'.
noVNC.js:7526 >> RFB.disconnect
noVNC.js:7526 >> Keyboard.allKeysUp
noVNC.js:7526 << Keyboard.allKeysUp
noVNC.js:7526 << RFB.disconnect
noVNC.js:7526 New state 'disconnected', was 'disconnecting'.
noVNC.js:7526 Clearing disconnect timer
noVNC.js:7526 << WebSock.onclose

x11vnc has no logs, continues running after disconnect

[anderoonies]

it appears that x11vnc might just not support extendeddesktopsize and i'm barking up the wrong tree

[anderoonies]

LibVNC/x11vnc#107

[samhed]

So basically x11vnc throws you out for sending a message it doesn't support? In that case, there's not much more we can do from our side.