Lub Hnov Dhia Dhia Jack, Google Coral TPU Accelerator Version: 4 Cov Kauj Ruam

2024 Tus sau: John Day | [email protected]. Kawg hloov kho: 2024-01-30 09:26

Nws txav nws txhais ceg, nws mloog koj cov lus xaj, nws tau tsav los ntawm kev siv tshuab tshiab tshaj plaws

Qhov "Hnov Dhia Dhia" yog qhov yooj yim siv tshuab hluav taws xob Jumping Jack, tsav los ntawm ob lub micro servos thiab lub iav yooj yim heev, muaj LEDs ua "qhov muag". Nws tau tswj hwm los ntawm cov lus txib yooj yim uas qhia txog cuaj txoj haujlwm ua ntej nws yuav tsum ua, lossis yog tias LED yuav tsum tau qhib lossis tua, lossis yog tias nws yuav tsum ua "ua las voos" ua ntej lossis teeb tsa kev txav chaw.

Lub hauv paus tseem ceeb ntawm cov txheej txheem yog Google Coral TPU tus tsav nrawm, uas tso cai rau khiav Tensorflow Lite qauv ua haujlwm offline nrog nrawm heev, txawm tias nyob ntawm lub tshuab "tsis muaj zog" zoo li Raspberry Pi. Qhov no tso cai xws li kev txheeb xyuas cov khoom nrawm thiab cais tawm siv RPi lub koob yees duab, tab sis kuj tseem siv lub tshuab kev kawm-raws lub suab paub ua haujlwm hauv zos.

Rau kuv qhov kev paub no yog thawj qhov kev tshaj tawm piv txwv rau Coral Accelerator lub suab nrhiav pom-tsav lub cev DIY lub cuab yeej, thiab cov piv txwv txuas ntxiv kuj tseem yuav siv rau lwm qhov, cov haujlwm nyuaj dua.

Kev tswj lub suab yog ua raws li piv txwv “hnov lus nab” hauv “phiaj xwm phiaj xwm phiaj xwm” (https://github.com/google-coral/project-keyword-spotter) uas tsis ntev los no (Cuaj hlis 2019) tau muab tso rau ntawm GitHub. Hauv kuv kev teeb tsa, cov txheej txheem suav nrog Raspberry Pi 4 nruab nrog Adafruit 16 channel servo lub kaus mom, Google Coral TPU Accelerator thiab lub vev xaib, ntawm no siv ua lub microphone. Jumping Jack tau piav qhia ua ntej hauv cov lus qhia yav dhau los, qhov uas nws tau tsav los ntawm Google Voice cov khoom siv los nyeem lub suab hais kom ua, txuas nrog rau Servo Bonnet hauv version 2.0 piav qhia hauv qab no.

Yav dhau los Google Voice Kit version muaj peb lub hauv paus txwv: nws tau nyob ntawm Google lub vev xaib raws li lub suab lees paub cov kev pabcuam thiab teeb tsa tau nyuab nyuab, nws yuav tsum tau nyem qee yam khawm ua ntej koj tuaj yeem hais kom ua, thiab muaj qhov qeeb qeeb nruab nrab ntawm hais cov lus txib thiab cov lus teb ntawm cov kab ke. Siv Google Coral accelerator txo cov lus teb rau vib nas this, tsis muaj kev cuam tshuam nrog kev siv internet thiab tau mloog txhua lub sijhawm. Nrog qee qhov kev hloov kho koj tuaj yeem siv nws los tswj cov cuab yeej nyuaj dua li Jumping Jack, raws li neeg hlau lossis tsheb, lossis txhua yam koj tuaj yeem tsim thiab tswj nrog Raspberry Pi.

Hauv nws cov lus tam sim no Cov Ntsiab Lus Tshawb Nrhiav nkag siab txog 140 lo lus tseem ceeb/kab lus tseem ceeb, tau piav qhia hauv cov qauv ua piv txwv ("suab_commands_v0.7_egetpu.tflite") thiab piav qhia hauv cov ntawv cais cais ("lab_gc2.raw.txt"). Txhais los ntawm cov ntaub ntawv hloov kho dawb ("commands_v2_hampelmann.txt"), cov ntsiab lus siv tshwj xeeb los ntawm peb tsab ntawv yog tom qab ntawd ua rau cov yuam sij ntawm lub keyboard virtual, piv txwv li. rau cov tsiaj ntawv, tus lej, nce/nqis/sab laug/sab xis, crtl+c, thiab lwm yam.

Tom qab ntawd, piv txwv li siv pygame.key, cov "keystrokes" no tau nyeem thiab siv los tswj xyuas qhov kev ua haujlwm ntawm lub cuab yeej, ntawm no qhov dhia dhia, yuav tsum ua. Hauv peb qhov xwm txheej no txhais tau tias yuav tsav ob lub servos rau txoj haujlwm uas tau npaj ua ntej, lossis tig LEDs rau lossis tua. Raws li cov ntsiab lus pom tus neeg sau nyob rau hauv kev cais cais, nws tuaj yeem mloog mus tas li rau koj cov lus xaj.

Version Cuaj hlis 21, 2019

Khoom siv

Raspberry Pi 4, ntawm Pimoroni

Google Coral TPU Accelerator, ntawm Mouser Germany, 72 €

Adafruit 16 Servo Bonnet, ntawm Pimoroni, txog 10 €

www.adafruit.com/product/3416

learn.adafruit.com/adafruit-16-channel-pwm…

Stacker header (yog xav tau)

www.adafruit.com/product/2223

4x AA roj teeb pob (lossis lwm lub zog 5-6V) rau Servo Bonnet

Lub webcam qub, zoo li lub microphone

Servo tsav Jumping Jack, raws li tau piav qhia yav dhau los. Cov txheej txheem teeb tsa tau txuas rau qib tom ntej, tab sis yuav xav tau kev hloov kho.

Yam yuav tsum tau ua rau Jumping Jack:

- 3 hli Forex phaj

- 2 micro servos

- 2 thiab 3 hli ntsia hlau thiab txiv ntseej

- 2 LEDs dawb thiab ib tus tiv thaiv

- me ntsis ntawm kab

Kauj ruam 1: Teem Up Device

Txhawm rau tsim Jumping Jack, thov ua raws cov lus qhia hauv kab lus qhia yav dhau los. Kuv siv Forex rau kuv tus qauv, tab sis koj tuaj yeem siv laser txiav acrylic lossis plywood phaj. Tej zaum koj yuav tsum tau kho qhov txheej txheem raws li qhov loj ntawm koj cov servos thiab lwm yam Kev ntsuas yog tias cov ceg thiab cov iav txav tau yam tsis muaj kev sib txhuam.

Teeb koj li Raspberry Pi. Ntawm Coral Github qhov chaw, muaj cov duab Raspian muaj uas muaj txhua yam uas yuav tsum tau ua kom khiav Coral nrawm ntawm Pi thiab muaj ntau txoj haujlwm, nrog txhua qhov teeb tsa twb tau ua lawm.

Tau txais cov phiaj xwm cov ntsiab lus pom los ntawm Google Coral GitHub nplooj ntawv. Nruab tag nrho cov software xav tau raws li qhia.

Nruab cov ntaub ntawv muab. Muab cov ntawv dhia dhia nab nab nab tso rau hauv txoj haujlwm cov ntsiab lus tseem ceeb cov ntawv nplaub tshev thiab cov ntsiab lus tseem ceeb cov ntaub ntawv hauv kev teeb tsa subfolder.

Txuas Adafruit Servo Bonnet rau Pi. Raws li kuv tab tom siv RPI vaj tsev nrog tus kiv cua, kuv xav tau siv GPIO stackers (piv txwv li muaj los ntawm Pimoroni) kom muaj kev sib txuas. Txhim kho txhua lub tsev qiv ntawv xav tau, raws li qhia hauv Adafruit cov lus qhia rau lub kaus mom hlau.

Txuas lub zog hluav taws xob 5-6V rau lub qhov hluav taws xob. Txuas cov servos thiab LEDs. Kuv qhov xwm txheej, Kuv siv chaw nres nkoj 0 rau LEDs thiab chaw nres nkoj 11 thiab 15 rau cov kev pabcuam.

Txhawm rau txheeb xyuas txhua yam, Kuv xav kom sim ua qhov phiaj xwm lo lus tseem ceeb "hnov nab" piv txwv thiab Adafruit servo lub kaus mom piv txwv ua ntej.

Kauj Ruam 2: Khiav Jumping Jack

Yog tias txhua ntu tau teeb tsa thiab ua haujlwm, sim siv nws. Koj tuaj yeem sau tsab ntawv hauv IDE lossis los ntawm kab hais kom ua.

Kev qw "txoj haujlwm 0" mus rau "txoj haujlwm 9" yuav ua rau Jumping Jack coj mus rau ib qho ntawm txoj haujlwm ua ntej. Kuv txhais "1" raws li ob txhais caj npab nce (uu), "3" raws li sab laug, sab xis (ud), "9" raws li ob txhais caj npab nqes (dd) thiab "5" raws li ob txhais caj npab nyob nruab nrab (cc).

uu uc ud = 1 2 3

ua cd = 4 5 6

ob dc dc: 7889

"0" zoo ib yam rau "5". "3" thiab "8" tsis tau lees paub zoo los ntawm lub ntsiab lus tseem ceeb thiab tej zaum yuav tsum tau rov ua dua.

Koj yuav tsum tau kho qhov tsawg kawg thiab qhov siab tshaj plaws rau txhua qhov servo/sab kom cov servos yuav tsis raug thaiv thiab tom qab ntawd rub lub zog ntau dhau.

"kev ua si tom ntej" yuav pib "seev cev", piv txwv li tau piav qhia ua ntu zus ntawm txoj haujlwm, thaum "kev sib tw ua si" yuav pib Dhia Nkaus kom ua ntu zus ntawm kev txav chaw. Hauv ob qho xwm txheej lawv yuav khiav mus ib txhis, yog li koj yuav tsum tso tseg kev txav mus los, piv txwv li. nrog "txoj haujlwm xoom" hais kom ua.

"nres game" yuav ua rau "ctrl + c" thiab nres cov txheej txheem.

"qhib rau" thiab "tua tawm" tuaj yeem siv tig LEDs rau thiab tawm.

Los ntawm kev hloov kho ntawm time.s pw qhov tseem ceeb koj tuaj yeem kho qhov nrawm ntawm kev txav mus los.

Kauj Ruam 3: Cov Cai thiab Cov Lus txib Cov Ntaub Ntawv

Cov cai nthuav tawm ntawm no yog kev hloov kho ntawm "hnov lus nab" txoj cai uas yog ib feem ntawm txoj haujlwm cov ntsiab lus tseem ceeb pob. Kuv tsuas yog tshem tawm txhua yam uas tsis tsim nyog rau kuv daim ntawv thov, yam tsis muaj kev nkag siab tiag ntawm cov ntsiab lus. Txhua yam kev txhim kho tau txais tos.

Tom qab ntawd kuv tau ntxiv ntu uas xav tau rau Adafruit Servo Bonnet, raws li lawv cov ntaub ntawv piv txwv.

Kuv xav ua tsaug rau cov programmer ntawm ob ntu.

Cov cai tuaj yeem pom txuas nrog ua ntawv. Siv nws ntawm koj tus kheej txoj kev pheej hmoo, hloov kho nws, txhim kho nws, ua si nrog nws.

# Copyright 2019 Google LLC

# # Muaj ntawv tso cai raws li Apache License, Version 2.0 ("License"); # koj yuav tsis siv cov ntawv no tshwj tsis yog ua raws li Daim Ntawv Tso Cai. # Koj tuaj yeem tau txais daim ntawv tso cai ntawm # # https://www.apache.org/licenses/LICENSE-2.0 # # Tshwj tsis yog xav tau los ntawm txoj cai lij choj lossis pom zoo sau ntawv, software # faib raws li Daim Ntawv Tso Cai muab faib rau ntawm "AS IS" BASIS, #Tsis muaj kev lav phib xaub lossis xwm txheej ntawm txhua tus menyuam, txawm hais tawm lossis cuam tshuam. # Saib Daim Ntawv Tso Cai rau cov lus tshwj xeeb tswj hwm kev tso cai thiab # cov kev txwv hauv qab Daim Ntawv Tso Cai. los ntawm _future_ ntshuam meej_import los ntawm _future_ ntshuam faib los ntawm _future_ ntshuam print_function ntshuam ntshuam ntshuam os los ntawm kev nkag tsis raug randint los ntawm xov ntshuam ntshuam ntshuam ntshuam sijhawm los ntawm edgetpu.basic.basic_engine ntshuam BasicEngine ntshuam qauv ntshuam pygame los ntawm pygame.locals ntshuam * ntshuam queue los ntawm random import Randrange los ntawm adafruit_servokit ntshuam ServoKit import board ntshuam busio ntshuam adafruit_pca9685 lub sijhawm ntshuam i2c = busio. I2C (board. SCL, board. SDA) lub kaus mom = adafruit_pca9685. PCA9685 (i2c) lub kaus mom.frequency = 60 pob = ServoKit (raws = 16) # teeb tsa tus lej ntawm cov channel #kit.servo [0].actuation_range = 160 #kit.servo [0].set_pulse_width_range (1000, 2000) #nce, nruab nrab thiab nqis nqis rau sab laug thiab sab caj npab up_l = 35 md_l = 90 dn_l = 160 up_r = 160 md_r = 90 dn_r = 35

lft = 15 # tus naj npawb ntawm qhov chaw nres nkoj servo, sab laug servo (0-8)

rgt = 11 # tus naj npawb ntawm qhov chaw nres nkoj servo, txoj cai servo (0-8) led_channel_0 = hat.channels [0] # LED teeb ntawm chaw nres nkoj 0 led_channel_0.duty_cycle = 0 # tig rau LED 100% # npe ntawm caj npab teeb tsa rau cuaj txoj haujlwm = [(md_l, md_r), (up_l, up_r), (up_l, md_r), (up_l, dn_r), (md_l, up_r), (md_l, md_r), (md_l, dn_r), (dn_l, up_r), (dn_l, md_r), (dn_l, dn_r)] # txhais 9 txoj haujlwm JumpingJack, qhia los ntawm cov lej 0-9 dance1 = (0, 8, 7, 4, 1, 2, 3, 6, 9, 8, 5, 2, 1, 4, 7, 8, 9, 6, 3, 2, 0) # a "dance" class Controler (object): #Callback function def _init _ (self, q): self._q = q def callback (self, hais kom ua): self._q.put (hais kom ua) chav Kawm App: def _init _ (self): self._running = True def on_init (self): pygame.init () self.game_started = True self._running = True return True def on_event (tus kheej, xwm txheej): yog event.type == pygame. QUIT: self._running = Cuav def JumpingJack0 (tus kheej, tuav): # tswj kev dhia Jack, cov ntsiab lus: "txoj haujlwm x" qhov tseem ceeb = int (yawm sij) p = txoj haujlwm [key] a = p [0] b = p [1] print ("Txoj hauj lwm:", qhov tseem ceeb, "sab laug /txoj cai: ", a,"/", b," degree ") # sys.stdout.write (" Txoj haujlwm: ", qhov tseem ceeb," sab laug/sab xis: ", a,"/", b," degree ") kit.servo [lft].angle = kit.servo [rgt].angle = b time.sleep (0.1) def JumpingJack1 (tus kheej): # tswj kev dhia dhia Jack, lo lus tseem ceeb: "game tom ntej" dnce = dance1 sp = (len (dnce)) rau r hauv thaj tsam (sp): #dance kev txiav txim ntawm txoj haujlwm, sp cov kauj ruam dc = dnce [r] yog (dc tsis nyob hauv thaj tsam (10)): # luam tawm ("yuam kev yuam kev ntawm txoj haujlwm", sp) dc = 4 p = position [dc] a = p [0] b = p [1] kit.servo [lft].angle = a kit.servo [rgt].angle = b time.sleep (0.25) # teeb nrawm ntawm kev txav mus dhia JumpingJack2 (tus kheej, tus yuam sij): # tswj Dhia Jack LEDs, lo lus tseem ceeb: "qhib/tua" coj = int (tuav) yog coj == 1: led_channel_0.duty_cycle = 0xffff # tig rau LED 100% lub sijhawm. (0.1) yog coj == 0: led_channel_0.duty_cycle = 0 # tua lub sijhawm LED.sleep (0.1) yog coj == 2: # ntsais led_channel_0.duty_cycle = 0xffff #turn rau LED 100% time.sleep (0.5) led_channel_0.duty_cycle = 0 #turn on LED 100% time.sleep (0.5) coj_channel_0.duty_cycle = 0xffff #turn ntawm LED 100% lub sijhawm.sleep (0.5) led_channel_0.duty_cycle = 0 #turn ntawm LED 100% time.sleep (0.5) led_channel_0.duty_cycle = 0xffff #turn ntawm LED 100% time.sleep (0.1) def JumpingJack3 (tus kheej): # tswj kev dhia Jack las voos, lo lus tseem ceeb: "game random" # rau h hauv thaj tsam (10): dr = randrange (9) p = txoj haujlwm [dr] a = p [0] b = p [1] kit.servo [lft].angle = a kit.servo [rgt].angle = b time.sleep (0.25) # teeb tsa qhov nrawm ntawm kev txav def spotter (tus kheej, args): cav = BasicEngine (args.model_file) mic = args.mic yog args.mic yog Tsis muaj lwm tus int (args.mic) qauv.classify_audio (mic, cav, lab_file = "teeb tsa/ntawv cim npe_gc2.raw.txt", commands_file = "config/commands_v2_hampelmann.txt", dectection_callback = self._controler.callback, sample_rate_hz = int (args.sample_rate_hz), num_frames_hop = int (args.num_frames_hop))

def on_execute (tus kheej, args):

yog tsis self.on_init (): self._running = cuav q = model.get_queue () self._controler = Controler (q) yog tsis args.debug_keyboard: t = Xov (target = self.spotter, args = (args,)) t.daemon = T.start () yam = -1 thaum tus kheej._running: pygame.event.pump () yog args.debug_keyboard: yuam sij = pygame.key.get_pressed () lwm tus: sim: new_item = q.get (Tseeb, 0.1) tshwj tsis yog kab ntawv. Empty: new_item = Tsis muaj yog tias new_item tsis yog Tsis muaj: yam = new_item yog (args.debug_keyboard thiab cov yuam sij [pygame. K_ESCAPE]) lossis cov khoom == "nres": self._running = Cuav # yog (args.debug_keyboard thiab cov yuam sij [pygame. K_SPACE]) lossis cov khoom == "mus": # tus kheej. JumpingJack0 (7) # yog (args.debug_keyboard thiab cov yuam sij [pygame. K_RIGHT]) lossis khoom == "yog": nws tus kheej. JumpingJack0 (6) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_LEFT]) lossis khoom == "sab laug": tus kheej. JumpingJack0 (4) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_UP]) lossis khoom == " nce ": tus kheej. JumpingJack0 (1) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_DOWN]) lossis khoom ==" nqis ": self. JumpingJack0 (9) yog (args.debug_keyboard thiab cov yuam sij [pygam e. K_0]) lossis khoom == "0": self. JumpingJack0 (0) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_1]) lossis khoom == "1": self. JumpingJack0 (1) yog (args. debug_keyboard thiab cov yuam sij [pygame. K_2]) lossis khoom == "2": self. JumpingJack0 (2) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_3]) lossis khoom == "3": self. JumpingJack0 (3) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_4]) lossis cov khoom == "4": self. JumpingJack0 (4) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_5]) lossis khoom == "5": nws tus kheej. JumpingJack0 (5) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_6]) lossis khoom == "6": tus kheej. JumpingJack0 (6) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_7]) lossis khoom == "7 ": self. JumpingJack0 (7) yog (args.debug_keyboard thiab cov yuam sij [pygame. K_8]) lossis khoom ==" 8 ": self. JumpingJack0 (8) yog (args.debug_keyboard thiab tuav [pygame. K_9]) lossis khoom == "9": self. JumpingJack0 (9) if (args.debug_keyboard and keys [pygame. K_a]) or item == "d": self. JumpingJack1 () #dance Jack, ntawm "next_game" yog (args. debug_keyboard thiab cov yuam sij [pygame. K_j]) lossis khoom == "j": self. JumpingJack2 (0) #LED rau, ON " switch_on "yog (args.debug_keyboard thiab cov yuam sij [pygame. K_k]) lossis cov khoom ==" k ": tus kheej. JumpingJack2 (1) #LED tawm, ntawm" swithch off "yog (args.debug_keyboard thiab cov yuam sij [pygame. K_l]) lossis khoom == "l": tus kheej. JumpingJack2 (1) #LED ntsais muag "lub hom phiaj" yog (args.debug_keyboard thiab cov yuam sij [pygame. K_r]) lossis khoom == "r": tus kheej. JumpingJack3 () #cov nkauj seev cev "random game" time.sleep (0.05) self.on_cleanup () yog _name_ == '_main_': parser = argparse. ArgumentParser () parser.add_argument ('-debug_keyboard', pab = 'Siv cov keyboard los tswj JumpingJack.

Kuj tseem muaj qhov hais kom ua teeb tsa cov ntawv "commands_v2_hampelmann.txt". Hloov kho raws li koj nyiam. Nws tsuas yog cov npe ntawm "hais kom ua, qhov tseem ceeb, (lub zog,)" ua ke, raws li daim ntawv lo-ntawv.

txoj haujlwm_zero, 0, position_one, 1, position_two, 2, position_three, 3, position_four, 4, position_five, 5, position_six, 6, position_seven, 7, position_eight, 8, position_nine, 9, move_up, up, go_up, up, move_down, down, go_down, nqis, txav_backwards, sab laug, txav_forwards, sab xis, rov_ rov qab, sab laug, mus_forwards, sab xis, 0.8 lub hom phiaj, l, ntsiag to, z, yog, y, tsis yog, n, hloov_on, j, hloov_off, k, ntim_up, nce, ntim _ down, nqis, next_game, d, random_game, r, start_game, s, stop_game, ctrl+c,

Kauj Ruam 4: Cov Tswv Yim Ntxiv thiab Lwm Yam Piv Txwv

Nws yog qhov pom tseeb tias qhov teeb tsa no kuj tseem tuaj yeem siv los tswj cov neeg hlau lossis lwm yam khoom siv. Yeej txhua yam uas yuav raug tswj los ntawm Raspberry Pi.

Kuv tab tom ua haujlwm txuas ntxiv ntawm tsab ntawv los tsav MeArm, thiab vam tias yuav tuaj yeem nthuav tawm qhov no thaum Lub Kaum Hli 2019.

Kuv tseem tab tom txiav txim siab siv Jumping Jack ua semaphore, thiab siv "txoj haujlwm posenet" txoj haujlwm taw tes taw kom paub ua qhov cuab yeej los nyeem Jumping Jack txoj haujlwm thiab txhais nws rov qab rau tus lej. Txoj kev no nws tseem tuaj yeem sib txuas lus ntawv, muab 2x 8 txoj haujlwm tuaj yeem qhia 64 tus lej sib txawv, ntau dua li txaus rau cov tsiaj ntawv, lej thiab cim. Qhov no tuaj yeem ua haujlwm, thaum hloov kho me ntsis, ua tiav lub cev rau kev thov IETF "Kev xa cov IP Datagrams hla Semaphore Chij Signaling System (SFSS)" (https://tools.ietf.org/html/rfc4824).

Tab sis qhov no yuav yog lwm qhov kev qhia. Thiab, raws li thawj qhov kev sim qhia tias dhia dhia yuav xav tau kev hloov pauv tseem ceeb ua ntej nws yuav raug lees paub tias yog tib neeg los ntawm AI cov txheej txheem yuav xav tau qee lub sijhawm.

Kuv xav kos koj lub ntsej muag rau cov lus qhia hauv qab no: Lub Hom Phiaj-Nrhiav-Tus Kheej-Pabcuam-Neeg Hlau-Ft-Raspberry, qhov twg ib qho khoom siv nrhiav neeg hlau siv ua ke ntawm Raspberry Pi thiab Google Coral TPU tau piav qhia.