inspiration source
I saw an interesting video at station B:
You can have a look. It's very interesting.
The up master reads the barrage content in the live broadcast room in real time through the code, then controls his computer, translates the barrage into instructions, and controls the game cyberpunk 2077.
There were more and more spectators, and finally even broke the direct room (of course, it was because the whole station of station B collapsed that day).
I'm very curious about how it was done.
Laymen watch the excitement while experts watch the doorway. As half an expert, we imitate the idea of the UP master and make one ourselves.
So my goal today is to reproduce a code that controls the live studio through the barrage, and finally start broadcasting in my own live studio.
Let me show you my final video:
[station B] imitate the UP master and make a live broadcast room controlled by barrage!
Does it look decent.
Design idea of the first edition
First, plan a general idea in your mind, as shown in the figure below:
This idea seems very simple, but we still have to explain. First, we need to find out how to catch the content of the barrage.
In most of our common live broadcast platforms, on the browser side, the barrage is pushed to the audience through WebSocket. In the mobile tablet and other clients (non Web clients), there may be some more complex TCP to push the barrage.
There is a good article about TCP message delivery, which is meituan's: Evolution of meituan terminal message delivery service Pike
In the final analysis, these barrages are realized by establishing long links between the client and the server.
Therefore, what we need to do is to use the code as the client and make a long link with the live broadcast platform. So we can get the barrage.
We just need to realize the whole barrage control process, so the capture of Barrage is not the focus of this paper. Let's find a ready-made wheel! After a search on Github, I found a very good open source library, which can obtain bullet screens of many live platforms:
https://github.com/wbt5/real-url
Tiktok and Kwai beep, and the bullet screen comments, the live streaming media and live barrage of 58 live platforms, which can be played in PotPlayer, flv.js and other players.
We clone the code, run the main function, and enter an address of the Bilibili live broadcasting room to get the real-time barrage stream of the live broadcasting room:
In the code, the obtained bullet screens (including user names) are printed directly on the console.
How did he do it? The Core Python code is as follows (unfamiliar with Python? It doesn't matter, it's regarded as pseudo code, which is easy to understand):
wss_url = 'wss://broadcastlv.chat.bilibili.com/sub'
heartbeat = b'\x00\x00\x00\x1f\x00\x10\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x5b\x6f\x62\x6a\x65\x63\x74\x20' \
b'\x4f\x62\x6a\x65\x63\x74\x5d '
heartbeatInterval = 60
@staticmethod
async def get_ws_info(url):
url = 'https://api.live.bilibili.com/room/v1/Room/room_init?id=' + url.split('/')[-1]
reg_datas = []
async with aiohttp.ClientSession() as session:
async with session.get(url) as resp:
room_json = json.loads(await resp.text())
room_id = room_json['data']['room_id']
data = json.dumps({
'roomid': room_id,
'uid': int(1e14 + 2e14 * random.random()),
'protover': 1
}, separators=(',', ':')).encode('ascii')
data = (pack('>i', len(data) + 16) + b'\x00\x10\x00\x01' +
pack('>i', 7) + pack('>i', 1) + data)
reg_datas.append(data)
return Bilibili.wss_url, reg_datasIt is connected to Bilibili's live barrage WSS address, that is, WebSocket address, and then disguised as a client to accept barrage push.
OK, after completing the first step, the next step is to send the barrage with a message queue. Open a separate consumer receiving barrage.
In order to make the implementation as simple as possible, instead of those professional message queues, redis's list is used as the queue to put the bullet screen content in.
The sender's core code is as follows:
# Link Redis
def init_redis():
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
return r
# message sender
async def printer(q, redis):
while True:
m = await q.get()
if m['msg_type'] == 'danmaku':
print(f'{m["name"]}: {m["content"]}')
list_str = list(m["content"])
print("Barrage splitting:", list_str)
for char in list_str:
if char.lower() in key_list:
print('Push queue:', char.lower())
redis.rpush(list_name, char.lower())After sending the barrage content, you need to write a consumer to consume these barrages and extract the instructions inside.
Moreover, after consumers receive the barrage, how to consume? We need a way to control the computer with code instructions.
Based on the principle of not making wheels, I found a Python automation control library PyAutoGUI
PyAutoGUI is a cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.
After installing this library and introducing it into the code, you can control the computer mouse and keyboard to perform corresponding operations through its API. It's perfect!
The Core Python code of the consumer (control computer) is as follows:
# Link Redis
def init_redis():
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
return r
# consumer
def control(key_name):
print("key_name =", key_name)
if key_name == None:
print("No instruction is issued this time")
return
key_name = key_name.lower()
# Control computer instructions
if key_name in key_list:
print("Issue instructions", key_name)
pyautogui.keyDown(key_name)
time.sleep(press_sec)
pyautogui.keyUp(key_name)
print("End instruction", key_name)
if __name__ == '__main__':
r = init_redis()
print("Start listening for barrage messages, loop_sec =", loop_sec)
while True:
key_name = r.lpop(list_name)
control(key_name)
time.sleep(loop_sec)ok, it's done. We open the barrage sending queue and consumers, and the queue of continuous circular consumption begins to run. Once there are wsad buttons in the barrage to control the game, the computer will issue instructions to itself.
Problems in the operation of the first edition
I excitedly opened my live studio of station B and started debugging. As a result, I found that I was still too naive. This first version of the code exposed a lot of problems. Let's talk about what the problem is and how I solve it.
Instructions are not user-friendly
Water friends actually like to send repeated words (overlapping words) such as www dddd, but the implementation of the first version only supports a single subtitle. Water friends found it ineffective and useless, so they left the live studio.
This is easy to solve. The bullet screen content is divided into each word and then pushed to the queue.
Solution: disassemble the barrage, disassemble the DDD into D,D,D, and send it to a consumer.
Hazard directive
The first is that the player's instructions are out of range.
When I opened the cyberpunk game and let the bullet screen audience control the driving in the game, a mysterious audience entered the live broadcasting room, silently sent an "F", and then...
Then the V in the game got out of the car. Gan, I asked you to drive, not to fight with the police...
Solution: add a barrage filter.
# Split the bullet screen and send only the specified instructions to consumers
key_list = ('w', 's', 'a', 'd', 'j', 'k', 'u', 'i', 'z', 'x', 'f', 'enter', 'shift', 'backspace')
list_str = list(m["content"])
print("Barrage splitting:", list_str)
for char in list_str:
if char.lower() in key_list:
print('Push queue:', char.lower())
redis.rpush(list_name, char.lower())After the above two problems are solved, the sender runs as follows:
Barrage command accumulation
This is a big problem. If we deal with all the barrage instructions sent by all water friends, there will be a problem that we can't consume.
Solution: it takes a fixed time to deal with the barrage, and others are discarded.
if __name__ == '__main__':
r = init_redis()
print("Start listening for barrage messages, loop_sec =", loop_sec)
while True:
key_name = r.lpop(list_name)
# Take out only one instruction at a time, and then empty the list, that is, throw away other bullets in this time window!
r.delete(list_name)
control(key_name)
time.sleep(loop_sec)There was a delay between the launch of the barrage and the audience seeing the result
In the first video, you can also feel that it takes about 5 seconds from the audience's command to be seen by the audience. Among them, at least 3 seconds are the delay of webcast stream, which is difficult to optimize.
Remanufactured version
After a series of tuning and involving, our version is from V0.1 to V0.2. The tiger shed tears.
The following is the reconstructed structure diagram:
Postscript
After writing this project, I tried it many times in the live studio, and the experience has been infinitely close to the video of UP master at that time. I started broadcasting and hung there for a long time, but when the popularity was the highest, there were only 20 people, a few dozen bullet screens, and many were sent by me. I also hope that the audience can pull more people to play together. It backfired.
It can be concluded that I have to have fans before I can play, woo woo woo. If you don't mind, you can pay attention to my B station account, also known as man san dao sauce. I'll occasionally take the wind and send some interesting technical videos.
All the codes implemented in this article are open source on Github. You can try it in your own live studio:
https://github.com/qqxx6661/l...
I'm an engineer who moves bricks in Ali
Continuous updating of high-quality articles is inseparable from your likes, forwarding and sharing!
The only technical public official account of the whole network: a talk about the backend Technology