Jetson Nano Quadruped Robot Object Detection Tutorial: 4 Kauj Ruam

2024 Tus sau: John Day | [email protected]. Kawg hloov kho: 2024-01-30 09:27

Nvidia Jetson Nano yog cov khoom tsim tawm, uas suav nrog SoM (System ntawm Module) thiab pawg thawj coj saib xyuas siv. Nws yog lub hom phiaj tseem ceeb tshaj plaws rau kev tsim cov kab ke uas xav tau kev ua haujlwm siab rau tshuab kev kawm, tshuab lub zeem muag thiab daim ntawv thov ua video. Koj tuaj yeem saib cov ncauj lus ntxaws ntxaws rau nws ntawm kuv tus YouTube channel.

Nvidia tau sim ua Jetson Nano raws li tus neeg siv-phooj ywg thiab yooj yim los tsim cov phiaj xwm nrog rau qhov ua tau. Lawv txawm pib kawm me ntsis ntawm yuav ua li cas-tsim koj lub neeg hlau nrog Jetson Nano, hnub tom qab lub rooj tsavxwm tau pib. Koj tuaj yeem pom cov ntsiab lus hais txog qhov haujlwm no ntawm no.

Txawm li cas los kuv tus kheej muaj qee qhov teeb meem nrog Jetbot raws li txoj haujlwm:

1) Nws tsis yog EPIC txaus rau kuv. Jetson Nano yog pawg thawj coj uas txaus siab heev nrog lub peev xwm ua tau zoo thiab ua lub log yooj yim neeg hlau nrog nws tsuas yog zoo li yog… qhov tsis txaus ntseeg ua.

2) Kev xaiv kho vajtse. Jetbot xav tau qee yam khoom siv uas kim/tuaj yeem hloov nrog lwm txoj hauv kev - piv txwv li lawv siv joystick rau kev sib tham hauv xov tooj. Suab zoo li kev lom zem, tab sis kuv puas xav tau tus joystick los tswj tus neeg hlau?

Yog li, tam sim ntawd tom qab kuv tau txais kuv txhais tes ntawm Jetson Nano Kuv pib ua haujlwm ntawm kuv tus kheej txoj haujlwm, Jetspider. Lub tswv yim yog los theej tawm qhov yooj yim ua piv txwv Jetbot muaj, tab sis nrog ntau yam khoom siv khoos phis tawj thiab siv tau rau ntau yam haujlwm.

Kauj Ruam 1: Npaj Koj Cov Khoos Phis Tawj

Txog qhov haujlwm no kuv tau siv tus qauv ua ntej ntawm Zuri plaub -plaub tus neeg hlau, ua los ntawm Zoobotics. Nws tau pw ib puag ncig hauv peb lub tuam txhab lab rau lub sijhawm ntev. Kuv tau teeb nws nrog lub laser txiav ntoo ntoo rau Jetson Nano thiab lub koob yees duab mount. Lawv cov qauv tsim muaj tswv, yog li yog koj li Jetson Nano neeg hlau koj xav tsim qee yam zoo sib xws, koj tuaj yeem saib Meped project, uas yog plaub npaug zoo ib yam nrog qhib qhov tsim. Qhov tseeb, txij li tsis muaj leej twg muaj lub hauv paus rau Zuri lub microcontroller (Arduino Mega) hauv peb chav kuaj, kuv tau siv cov cai los ntawm Meped nrog qee qhov kev hloov me me hauv txhais ceg/taw taw.

Kuv siv niaj hnub USB Raspberry Pi tau tshaj lub web-cam thiab Wifi USB dongle.

Lub ntsiab lus tseem ceeb yog vim tias peb tab tom yuav siv Pyserial rau kev sib txuas lus txuas ntawm microcontroller thiab Jetson Nano, koj lub kaw lus tseem ceeb tuaj yeem siv txhua hom microcontroller, tsuav nws tuaj yeem cuam tshuam nrog Jetson Nano nrog USB txuas txuas. Yog tias koj tus neeg hlau siv DC lub cav thiab tus tsav tsheb (piv txwv li L298P-based) nws tuaj yeem ncaj qha cuam tshuam nrog tsav tsheb nrog Jetson Nano GPIO. Tab sis, hmoov tsis zoo, rau kev tswj hwm servos koj tsuas tuaj yeem siv lwm lub microcontroller lossis mob siab rau I2C servo tsav tsheb, txij li Jetson Nano tsis muaj kho vajtse GPIO PWM.

Los xaus, koj tuaj yeem siv hom neeg hlau nrog txhua lub microcontroller uas tuaj yeem txuas nrog Jetson Nano siv USB cov ntaub ntawv cable. Kuv tau tshaj tawm cov cai rau Arduino Mega rau github chaw cia rau qhov kev qhia no thiab ib feem cuam tshuam rau cuam tshuam nrog Jetson Nano nrog Arduino nyob ntawm no:

yog (Serial.available ()) {hloov (Serial.read ()) {

{

rooj plaub '1':

rau pem hauv ntej ();

tawg;

rooj plaub '2':

rov qab ();

tawg;

rooj plaub '3':

tig_right ();

tawg;

rooj plaub '4':

tig_left ();

tawg;

Peb tshawb xyuas yog tias muaj cov ntaub ntawv muaj, thiab yog tias muaj, hla nws mus rau qhov kev hloov pauv rooj plaub. Ua tib zoo saib, cov ntaub ntawv los ntawm cov ntawv tuaj ua tus cim, ceeb toom cov lus hais ib zaug ib ncig ntawm tus lej 1, 2, 3, 4.

Kauj Ruam 2: Txhim Kho Cov Ntim Ntim

Hmoov zoo rau peb, default Jetson Nano system cov duab los nrog ntau yam khoom ua ntej teeb tsa (zoo li OpenCV, TensorRT, thiab lwm yam), yog li peb tsuas xav tau nruab ob peb lwm pob khoom kom ua txoj haujlwm thiab pab kom SSH.

Cia peb pib los ntawm kev pab SSH yog tias koj xav ua tag nrho cov haujlwm uas nyob deb.

sudo apt hloov tshiab

sudo apt nruab openssh-server

SSH server yuav pib ua haujlwm.

Txhawm rau txuas rau koj lub tshuab Ubuntu dhau LAN koj tsuas yog yuav tsum nkag mus rau cov lus txib hauv qab no:

ssh username@ip_address

Yog tias koj muaj lub tshuab Windows, koj yuav tsum teeb tsa SSH tus thov kev pab, piv txwv li Putty.

Cia peb pib los ntawm txhim kho Python Pob Tus Thawj Tswj (pip) thiab Hauv ncoo rau kev ua duab.

sudo apt nruab python3-pip python3-pil

Tom qab ntawd peb yuav teeb tsa Jetbot chaw cia khoom, txij li peb tab tom tso siab rau qee qhov ntawm nws lub hauv paus los ua kom pom cov khoom.

sudo apt nruab python3-smbus python-pyserial

git clone

cd jetbt

sudo apt-tau nruab cmake

sudo python3 setup.py nruab

Thaum kawg clone kuv Github chaw cia rau qhov haujlwm no rau koj lub tsev ntawv thiab teeb tsa Flask thiab qee qhov lwm pob khoom rau neeg hlau lub chaw taws teeb tswj siv web server.

git clone

ua cd

sudo pip3 install -r requirements -opencv

Rub tawm SSD pretrained SSD (Ib Leeg Kuaj Ntes) tus qauv los ntawm qhov txuas no thiab muab tso rau hauv jetspider_demos nplaub tshev.

Tam sim no peb zoo mus!

Kauj ruam 3: Khiav Code

Kuv tau ua ob qhov kev nthuav qhia rau Jetspider, thawj qhov yog kev sib tham yooj yim, zoo ib yam uas kuv tau ua ua ntej rau Banana Pi rover thiab qhov thib ob siv TensorRT rau kev txheeb xyuas cov khoom thiab xa cov lus txib txav mus rau qhov txuas txuas mus rau microcontroller.

Txij li feem ntau ntawm cov lej xov tooj tau piav qhia hauv kuv lwm qhov kev qhia (Kuv tsuas yog ua qee qhov kev hloov me me, rov ua dua cov vis dis aus) ntawm no kuv yuav tsom mus rau Lub Hom Phiaj Tshawb nrhiav.

Cov ntawv tseem ceeb rau cov khoom hauv qab no yog object_following.py hauv jetspider_object_following, rau kev sib tham hauv xov tooj yog spider_teleop.py hauv jetspider_teleoperation.

Cov khoom hauv qab tsab ntawv pib nrog kev xa cov qauv tsim nyog thiab tshaj tawm cov kev hloov pauv thiab cov xwm txheej hauv chav kawm. Tom qab ntawd peb pib Flask web server nrog kab no

app.run (host = '0.0.0.0', threaded = Tseeb)

Sai li peb qhib 0.0.0.0 (localhost) chaw nyob hauv peb lub vev xaib lossis Jetson Nano chaw nyob ntawm lub network (tuaj yeem tshawb xyuas nrog ifconfig hais kom ua), txoj haujlwm no yuav ua tiav

def qhov ntsuas ():

Nws ua rau lub vev xaib nplooj ntawv peb muaj nyob hauv cov ntawv tais ceev tseg. Tus qauv muaj cov vis dis aus tso rau hauv nws, yog li thaum nws ua tiav qhov chaw thau khoom, def video_feed (): yuav ua tiav, uas xa cov lus teb uas tau pib nrog lub tshuab hluav taws xob ua haujlwm.

Qhov zais cia txhawm rau txhim kho hloov chaw (hloov kho cov duab hauv lub vev xaib rau peb cov vis dis aus) yog siv cov lus teb ntau. Cov lus teb ntau tus suav nrog lub taub hau uas suav nrog ib qho ntawm ntau hom ntsiab lus hom, ua raws los ntawm ntu, sib cais los ntawm tus cim ciam teb thiab txhua tus muaj nws tus kheej ib feem tshwj xeeb cov ntsiab lus.

Hauv def gen (): kev ua haujlwm peb siv lub tshuab hluav taws xob ua haujlwm hauv lub voj tsis muaj qhov kawg uas ntes cov duab, xa nws mus rau def execute (img): muaj nuj nqi, muab cov duab xa mus rau nplooj ntawv web tom qab ntawd.

def execute (img): kev ua haujlwm yog qhov uas txhua qhov khawv koob tshwm sim, nws siv daim duab, hloov pauv nws nrog OpenCV thiab hla nws mus rau Jetbot ObjectDetector chav kawm piv txwv "qauv". Nws xa rov qab cov npe ntawm kev kuaj pom thiab peb siv OpenCV los kos cov duab plaub plaub puag ncig lawv thiab sau cov lus piav qhia nrog cov khoom pom pom hauv chav kawm. Tom qab ntawd peb tshawb xyuas yog tias muaj ib qho khoom siv ntawm peb qhov kev txaus siab pom tau sib xws_detections = [d rau d hauv kev kuaj pom [0] yog d ['daim ntawv lo'] == 53]

Koj tuaj yeem hloov tus lej ntawd (53) mus rau lwm tus lej los ntawm CoCo cov ntaub ntawv yog tias koj xav kom koj tus neeg hlau ua raws lwm yam khoom, 53 yog cov kua. Tag nrho cov npe yog nyob rau hauv pawg.py cov ntaub ntawv.

Thaum kawg yog tias tsis muaj qhov ntsuas pom rau 5 vib nas this peb xa tus cim "5" rau neeg hlau kom nres hla cov ntawv. Yog tias pom cov khoom peb suav tias nws nyob deb npaum li cas los ntawm qhov nruab nrab ntawm cov duab thiab ua raws ((yog nyob ze rau nruab nrab, mus ncaj (cim "1" ntawm ntu), yog nyob sab laug, mus rau sab laug, thiab lwm yam). Koj tuaj yeem ua si nrog cov txiaj ntsig no los txiav txim siab qhov zoo tshaj plaws rau koj kev teeb tsa tshwj xeeb!

Kauj Ruam 4: Kev Xav Thaum Kawg

Nov yog lub ntsiab lus ntawm ObjectFollowing demo, yog tias koj xav paub ntau ntxiv txog Flask webserver Video streaming, koj tuaj yeem saib ntawm qhov kev qhia zoo no los ntawm Miguel Grinberg.

Koj tseem tuaj yeem saib Nvidia Jetbot Lub Hom Phiaj Tshawb Nrhiav phau ntawv ntawm no.

Kuv vam tias kuv qhov kev coj ua ntawm Jetbot kev nthuav dav yuav pab tsim koj tus neeg hlau siv Jetbot moj khaum. Kuv tsis tau siv txoj hauv kev tiv thaiv kev tiv thaiv kev tawm tsam, txij li kuv xav tias kev xaiv tus qauv yuav tsis ua rau muaj kev tiv thaiv kev tiv thaiv zoo.

Ntxiv rau kuv ntawm LinkedId yog tias koj muaj lus nug thiab sau npe yuav rau kuv tus YouTube channel kom tau txais kev ceeb toom txog cov phiaj xwm nthuav dav uas cuam tshuam nrog kev kawm tshuab thiab neeg hlau.