2
Ask HN: Reinforcement learning for single, lower end graphic cards?
source link: https://news.ycombinator.com/item?id=27872212
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Ask HN: Reinforcement learning for single, lower end graphic cards? Ask HN: Reinforcement learning for single, lower end graphic cards?
23 points by DrNuke 6 hours ago | hide | past | favorite | 7 comments On one side, more and more hardware is being thrown in parallel to ingest and compute astonishing amounts of data generated by realistic 3d simulators, especially for robotics, with big names like OpenAI now just giving up on the field as from https://news.ycombinator.com/item?id=27861201 ; on the other side, more recent simulators like Brax from Google https://ai.googleblog.com/2021/07/speeding-up-reinforcement-learning-with.html are aiming at “matching the performance of a large compute cluster with just a single TPU or GPU”. Where do we stand on the latter side of the equation then? What is the state of the art with single, lower end GPUs like my 2016 gaming laptop’s GTX 1070 8GB? What do we lower end users need to read, learn and test these days? Thanks.
Apply early for the YC Winter 2022 batch
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Search:
For many RL problems you don't really need GPUs because the networks used are relatively simple compared to supservised learning, and most simulations are CPU-bound. Many RL problems are constrained by data so that running simulations (CPU) is the bottleneck, not the network.
Agreed, deep architectures are really only needed for feature engineering. There have been a few papers showing that even for these very deep setups, the actual policy can almost always be fully captured in a small mlp.
Can you share some recent references?
(Are you referring to the early papers showing that MPC and LQR solve SOME problems faster ?!)
This is mostly in the realm of computer vision, but I would recommend checking out AlexeyAB's fork of Darknet: https://github.com/AlexeyAB/darknet It's got decent CUDA acceleration, I personally run a GTX 960M for training.
Instead of using your low end GPU, you could get a TPU like https://coral.ai/docs/edgetpu/benchmarks/. Or rent a single GPU on the cloud which costs less than a $/hour and can be free in certain cases.
In terms of APIs, you can try WebGPU which is nominally meant for Javascript in the browser, but there are native interfaces for it such as Rust: https://github.com/gfx-rs/wgpu
Check out Andrej karpathys convnet.js and deepq learning web apps.
Convnet.js hasn't been updated in over half a decade.
Apply early for the YC Winter 2022 batch
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Search:
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK