Overview
- Scalable Socket.IO Application
- Extent Event Middleware
- Performance Best Practices
- Sticky-Session Load Balancing
Scalable Socket.IO Application
Share Sockets In Multi-Nodes
Socket.IO relied on WebSocket protocol which use persistent TCP connection between client and a server. In a distributed system, Redis-Adapter is used to maintain pool of sockets which are distributed in multi-nodes.
With Socket.IO 4.0, A arbitrary object socket.data & io.fetchSockets() are introduced which can access distributed socket and shared data in multi-nodes.
// server A
io.on("connection", (socket) => {
socket.data.username = "alice";
});// server B
const sockets = await io.fetchSockets();
console.log(sockets[0].data.username); // "alice"
Join Broadcast Room To Each Device / Tab Of A Given User
A user may have many socket connections by using browser multi-tabs. For associates each socket of a given user, we can join user’s sockets to a room for broadcasting events. Use io.in() to broadcast event to all user’s devices/tabs.
io.on("connection", (socket) => {
const { user } = socket.request; socket.join(user.id); // bind all devices of an user to a room io.in(user.id).emit(
"user.event.connected",
{
"userId": user.id,
"message": `${user.username} are connected.`
},
);
});
Extend Event Middleware
Socket.IO provide handshake middleware io.use() and global event middleware socket.use(). socket.use() will capture all incoming request and go through. However, Socket.IO lack of support for specific event middleware which let developers are hard to organize complex business logic.
Therefore, we implement specific event middleware socket.use(event, fns) by overwriting Socket.prototype.use & Socket.prototype.run.
const use = Socket.prototype.use;Socket.prototype.use = function (...args) {
const [eventName, fns] = args; if (typeof eventName === "function") {
return use.call(this, eventName);
} // support socket.use(eventName, fns)
if (typeof eventName === "string") {
if (this.__fns === undefined) {
this.__fns = {};
}
this.__fns[eventName] = fns;
} return this;
};Socket.prototype.run = function (event, fn) {
const fns = this.fns.slice(0);
const __fns = this.__fns && this.__fns[event[0]]; // merge combine global & event socket middlewares.
if (Array.isArray(__fns) && __fns.length) {
fns.push(...__fns);
} if (!fns.length)
return fn(null); function run(i) {
fns[i](event, function (err) {
// upon error, short-circuit
if (err)
return fn(err);
// if no middleware left, summon callback
if (!fns[i + 1])
return fn(null);
// go on to next
run(i + 1);
});
}
run(0);
};
And then, you can use event middleware like:
const preHandler = [
// middleware 1
function ([eventName, payload], next) {
// TODO
next();
},
// middleware 2
function ([eventName, payload], next) {
// TODO
next();
},
];const handler = function (payload) {
io.in(socket.request.user.id).emit(
"order.event.got"
{ id: 1 }
);
}socket
.use("order.command.get", preHandler)
.on("order.command.get", handler);
Performance Best Practices
Parsing/Serialization
Socket.IO default parser use JSON.parse() & JSON.stringify() to parsing/serialization. JSON.parse() & JSON.stringify() are Node.JS V8 Sync Function will block Node.JS event loop. Therefore, a good parser socket.io-msgpack-parser is performant to do parsing/serialization by using binary encoding/decoding.
Sticky-Session Load Balancing
We use k8s ingress-aginx as backend for AWS NLB infrastructure.
[client] -> [aws nlb] -> [ingress-nginx] -> [socket.io]Transports using websocket or polling
When we use websocket as transport, it relies on a single TCP connection for the whole session. We don’t need to configure sticky-session for routing requests to connected pod.
However, old browser is not support websocket which mean we need to fallback use HTTP long polling, that we should configure ingress sticky-session load balancing for this case.
AWS ELB Connection Idle Timeout
If you use AWS ELB (Classic), you should prevent connection idle timeout (60s) is not enough, it’s better to set > 360s.
If you use AWS NLB, the default connection timeout is 300 seconds is good enough.
Ingress-Nginx Connection Idle Timeout
proxy send/read/connection timeout default is not enough, it’s better to set > 360s.
Transports using websocket or polling
When we use websocket as transport, it relies on a single TCP connection for the whole session. We don’t need to configure ip hash/sticky session let connection/request to correct pod. However, when we use polling, we should configure ip hash/sticky session because HTTP protocol.
Sticky Session (polling configure required)
If we want to use sticky session. we need to add annotation to ingress.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: chat-ingress
namespace: chat
annotations:
kubernetes.io/ingress.class: "ingress-nginx-chat"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
rules:
- host: example.com
http:
paths:
- path: /socket.io
backend:
serviceName: chat-service
servicePort: 80IP Hash (polling configure required)
If you want to use ip hash to replace sticky session. We need AWS NLB & ingress-nginx to preserve client ip.
- Configure Load Balancer Service externalTrafficPolicy as Local.
- ConfigMap add data use-forwarded-headers: “true” & proxy-real-ip-cidr: 0.0.0.0/0.
Then, upstream header contain X-Forwarded-For which preserve client ip.
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: chat
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
data:
use-proxy-protocol: "false"
use-forwarded-headers: "true"
proxy-real-ip-cidr: "0.0.0.0/0" ---kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: chat
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-type: nlb # AWS CLB
# service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
spec:
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: http
externalTrafficPolicy: "Local"
Configure ingress use ip hash.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: chat-ingress
namespace: chat
annotations:
kubernetes.io/ingress.class: "ingress-nginx-chat"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
nginx.ingress.kubernetes.io/upstream-hash-by: "$x_forwarded_for"
spec:
rules:
- host: example.com
http:
paths:
- path: /socket.io
backend:
serviceName: chat-service
servicePort: 80
