Guest Posted January 19 Posted January 19 hi everybody... are you planning to migrate the IO scheduler to the NPU? in most cases, it is not involved on the devices that your operating system serves. This would give space for new perspectives.
going Posted January 19 Posted January 19 5 часов назад, Maksimarm сказал: are you planning to migrate the IO scheduler to the NPU? @Maksimarm What do you mean by that? Can you explain it in an expanded form?
Guest Posted January 20 Posted January 20 I think you understand what an NPU module is! on some SoCs, it is available but not used. why not use it to deploy the IO scheduler on it. the algorithms of the existing Linux kernel schedulers require serious improvements, so building a new algorithm on an unused NPU module would give the Armbian operating system a chance to reach a new level. You really need this right now.
Werner Posted January 20 Posted January 20 So this is more or less a feature request. Therefore moved.
Igor Posted January 20 Posted January 20 2 hours ago, Maksimarm said: so building a new algorithm on an unused NPU module would give the Armbian operating system a chance to reach a new level. This is open source world. Once we have it, everyone else will have it to. 2 hours ago, Maksimarm said: You really need this right now. To maximize our time, emotional and financial loss? We are already loosing tens of thousands of hours every year for what we provide now. Donations, as only source to cover this wish, supports our current work in around 0.5%, which means yearly budget for 2024 R&D is already long gone. Most of HW dealers and predatory users & competitors not just steal (take) in regular ways, but also generate damages however they can, for fun or to make profits for themselves. Re-thing your question. Before you suggest us what we (anyone that operates in open source) need, we we need something from general public: understanding and funding.
Guest Posted January 20 Posted January 20 you say yourself that the path you are on now does not give you anything. your only chance to change everything is startup on your Arbian_board with a unique processor and a unique operating system. you have the opportunity to take into account all the mistakes in both software and hw and give everyone what they are waiting for. if it works out, then you will stop being poor)))) if not, then you will live on donations.
going Posted January 20 Posted January 20 4 часа назад, Maksimarm сказал: I think you understand what an NPU module is! on some SoCs, it is available but not used. The devils are hiding in the details! Some hardware vendors claim that NPUs in their processors work on some operating systems. This is the OS on their cores. Which specific chip are you interested in?
going Posted January 20 Posted January 20 33 минуты назад, Maksimarm сказал: and give everyone what they are waiting for. And what is the user waiting for? Can you formulate that?
Igor Posted January 20 Posted January 20 1 hour ago, Maksimarm said: that the path you are on now does not give you anything. Never did. We sponsor your consumerism to the degree we want. 1 hour ago, Maksimarm said: and a unique operating system It seems you don't understand what open source is. 1 hour ago, Maksimarm said: and give everyone what they are waiting for Download what we give you and do whatever you want with that. 1 hour ago, Maksimarm said: if it works out, then you will stop being poor What is the definition of poor? Can poor people invest into software development that you can consume it for free? We are already investing millions into open source, but everyone has spending limits. Since you (public) will never understand yours, we have to stop your nonsense in some way. But if you finance, we can hire, if not, you can hire yourself and do whatever you want. If you succeed, we will download and use your idea. Welcome to open source world.
Guest Posted January 20 Posted January 20 (edited) you ask what people want, I tell you that it is certainly not the Armbian that you give them. you probably don't communicate with ordinary users on other forums. there is a lot of negativity about the Linux kernel. It's not that we should thank you for giving it this Linux kernel, it's you who should thank us for using it. we are testing it. Without us, this kernel would be useless to anyone. only we users make something popular and in demand. You're not doing this. we can like any shit and this shit will become a sought-after product. I hope you understand what I'm talking about. Good luck to you guys. Edited January 20 by Maksimarm
Igor Posted January 20 Posted January 20 2 hours ago, Maksimarm said: I hope you understand what I'm talking about. It is important that you understand what I am talking about. 2 hours ago, Maksimarm said: about the Linux kernel. We don't represent Linux kernel. Actually nobody does. Linux corporation https://www.linuxfoundation.org/ is the closest entity doing that and even they are considered small and poor in the corporate world they are in. What Armbian (a few people that maintains this place) does? We provide tools and build kernels with small but important changes focused into specific single board computers. We add valuable code that this generic kernel code (https://lwn.net/Articles/915435/) becomes usable. Sometimes only to some degree. And that's it. You want to develop something but you seek sponsors of your idea? No problem, just find someone else. We already burned all yearly budget. 2 hours ago, Maksimarm said: there is a lot of negativity Negativity is shaped by your (over)expectations. From our perspective, they are totally off the chart. Once you get it, you will perhaps even support us. Until then, you will treat us like sh* and we will both be unhappy.
ag123 Posted January 20 Posted January 20 here is some 2 cents comments, if you are meaning NPUs like these: https://github.com/rockchip-linux/rknpu2 - hardware interfaces are kept as trade secrets and not published anywhere 1st the hardware interfaces are practically undocumented, what they provide is mostly an 'sdk' with some binary blobs it practically means there is *no way* to use the NPU as those binary blobs in turn depend on device drivers which again are binary blobs (no source) and there is no hardware documentation any where about the technical details, registers etc. if that at least those are published, one could possibly start coding something to test things on the NPU. then that for things like ethos-n78 https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-n78 you can find some info here https://developer.arm.com/Processors/Ethos-N78 but that it seemed the real SOCs with that chip is no where to be seen let alone any boards found with them. - IO / cpu scheduling cpu frequency scaling / governors are well documented https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt https://docs.kernel.org/scheduler/schedutil.html https://www.kernel.org/doc/html/latest/scheduler/ and for 'simple' ARM (or RISC V) chips, any sort of 'elaborate' scheduling are probably going to just burn more cpu cycles with little to gain. but that nevertheless the source codes and the documentations are there so that one can try to develop your own governor if you prefer. and that the elaborate 'advanced' schedutil governor is already there in the kernel, likely in Armbian as well. Hence, one can proceed to improve that if one deem that the state-of-the-art currently isn't adequate. and if one wants to do some manual scheduling there is the plain old "nice" command https://www.scaler.com/topics/linux-nice/ ---- well my thoughts, scheduling and NPU are 2 unrelated issues, it is possible to handle 'elaborate' scheduling without an NPU, this is currently the state-of-the-art from the 'lowly' ARM boards that we are running, to top tier high core count Intel Xeon / AMD Epyc processors and running all that loads ranging from amazon, google, chatgpt etc, no issues. The other thing being the NPUs itself, currently many SOC IP owners, held their hardware interfaces as *trade secrets* and refused to release them. You would need to jump that hoop to even use the NPU without any documentation, or else use their proprietary binary blob software, which won't work outside their proprietary binary blob distributions. This withheld *trade secret* about the NPU is the biggest pitfall / trap to those buying those boards with those SOCs and wanting to use the NPU. you get *no support*, *no help*, *no nothing* after you buy the board which purportedly has the NPU. practically *useless*. don't even bother to try it for any 'test' 'AI' stuff, you may at best get a *binary blob demo* and that's it (and it is not anywhere close to even using it for any practical purpose, let alone scheduling). And much more than that, using an NPU practically means that your neural network model must be *quantized*, if you know what that means. All those small NPU hardware normally handles like 8 bit integers, 16 bit integers or at best 16 bit floats. This practically means that you would need to spend a lot of effort to *convert* even ordinary neural networks into the *quantized* form that can be processed by the NPU, if you can't convert that it is unusable. Even if you managed to convert that there is a risk of lost of precision, e.g. if you convert a 32 bit float down to an 8 bit int, you may practically be quantizing a number space of 4 billion numbers (actually more) to 256 quantized levels. that is the extreme of the information loss, and at the end of the day, if it even works, you may simply get *wrong* results, and again it is practically *useless*. 4
Guest Posted January 21 Posted January 21 18 hours ago, Igor said: We don't represent Linux kernel. but you are using the Linux kernel without improving it with all the flaws that are contained in the Linux kernel. There are a lot of them there. In general, your position is that of an ostrich with its head in the sand. I wanted to offer you something to change for free, to try to change it, but your reaction shocked me. We have nothing more to talk about. Your Armbian does not have a bright future in the Linux kernel.
Guest Posted January 21 Posted January 21 17 hours ago, ag123 said: And much more than that, using an NPU those who want to do something are looking for ways, those who don't want to are looking for excuses and reasons. you are not moving the world forward, you are pulling it back.
Gunjan Gupta Posted January 21 Posted January 21 @MaksimarmYou are evading the first ever question asked in reply to your thread, which is what you meant by migrating io scheduler to NPU. As I believe will be the case with most of us, when someone says NPU we will think about Neural Processing Units which are for use with AI inferences and hence something that will be used by end users. It has nothing to do with IO scheduler as explained by ag123 above as well. Hence we did asked you what you mean't by it as it does seem like you are clearly not talking about the same thing or have different understanding of the same.
Gunjan Gupta Posted January 21 Posted January 21 57 minutes ago, Maksimarm said: those who want to do something are looking for ways, those who don't want to are looking for excuses and reasons. you are not moving the world forward, you are pulling it back. Alright, lets move the world forward together. How about you book a session for consultation and pay for the same - https://calendly.com/armbian/consultation?month=2024-01. We can discuss about what needs to be done there. Then you fund the development and we will happily work on it maybe even fulltime depending upon how much you are willing to fund.
Igor Posted January 21 Posted January 21 2 hours ago, Maksimarm said: I wanted to offer you something to change for free You are delusional? Offer? From you? You asked a question about the topic you clearly don't understand well, @ag123 sponsored you answer / education / FYI and that's about it. 2 hours ago, Maksimarm said: but you are using the Linux kernel without improving it You clearly have no clue what we do. Do your homework. Insulting us is not going into your favor. 2 hours ago, Maksimarm said: those who want to do something are looking for ways, those who don't want to are looking for excuses and reasons. you are not moving the world forward, you are pulling it back. What are you waiting? If its free for you, invest your time. Ours is not free, so ... good luck.
ag123 Posted January 21 Posted January 21 some additional off-topic blurbs about NPU etc. https://www.intc.com/news-events/press-releases/detail/1663/intel-accelerates-ai-everywhere-with-launch-of-powerful Quote Dec 14, 2023 NEW YORK--(BUSINESS WIRE)-- At its “AI Everywhere” launch in New York City today, Intel introduced an unmatched portfolio of AI products to enable customers’ AI solutions everywhere — across the data center, cloud, network, edge and PC. Highlights include: The Intel® Core™ Ultra mobile processor family, the first built on the Intel 4 process technology and the first to benefit from the company’s largest architectural shift in 40 years, delivers Intel’s most power-efficient client processor and ushers in the age of the AI PC. The 5th Gen Intel® Xeon® processor family is built with AI acceleration in every core, bringing leaps in AI and overall performance and lowering total cost of ownership (TCO). Intel CEO Pat Gelsinger showed for the first time an Intel® Gaudi®3 AI accelerator, arriving on schedule next year. (so we are aalready there !) https://arstechnica.com/gadgets/2023/12/amds-new-ryzen-8040-laptop-chips-look-a-lot-like-the-ryzen-7040-cpus/ Quote AMD's briefing and slide deck spend most of their time talking about Ryzen AI and the dedicated NPU. Though Nvidia dominates the market for server-side AI chips, local NPU hardware from AMD, Intel, and various ARM chipmakers may make it easier to run some generative AI workloads locally. Depending on how those features are implemented, they could be better for privacy since the processing is happening on-device rather than sending data to a server somewhere—when Apple talks about privacy, it often talks about this kind of on-device processing since its chips have included NPUs for years longer than Intel's or AMD's. https://www.amd.com/en/technologies/xdna.html https://coral.ai/ https://developers.googleblog.com/2022/05/coral-googles-platform-for-edge-ai.html Quote Coral, Google’s platform for Edge AI, chooses ASUS as OEM partner for global scale ASUS IoT already has a long-standing history of collaboration with Coral, being the first partner to release a product using the Coral SoM when they launched the Tinker Edge T development board. ASUS IoT has also integrated Coral accelerators into their enterprise class intelligent edge computers and was the first to release a multi Edge TPU device with the award winning AI Accelerator PCIe Card. Because we have this history of collaboration, we know they share our strong commitment to new innovation in edge computing. https://www.nvidia.com/en-sg/ai-data-science/ Quote Artificial Intelligence The Most Advanced AI, Ready for Enterprise https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-n78 Quote NPU Ethos-N78 Highly Scalable and Efficient Second-Generation ML Inference Processor https://wiki.t-firefly.com/en/ROC-RK3588S-PC/usage_npu.html Quote 1. NPU RK3588 has a NPU(Neural Process Unit) that Neural network acceleration engine with processing performance up to 6 TOPS. Using this NPU module needs to download RKNN SDK which provides programming interfaces for RK3588S/RK3588 chip platforms with NPU. This SDK can help users deploy RKNN models exported by RKNN-Toolkit2 and accelerate the implementation of AI applications https://semiconductor.samsung.com/news-events/news/samsung-electronics-to-strengthen-its-neural-processing-capabilities-for-future-ai-applications/ Quote Samsung Electronics to Strengthen its Neural Processing Capabilities for Future AI Applications Samsung Electronics Co., Ltd., a world leader in advanced semiconductor technology, today announced that it will strengthen its neural processing unit (NPU) capabilities to further extend the reach of its artificial intelligence (AI) solutions. https://www.qualcomm.com/news/onq/2023/10/introducing-our-next-gen-intelligent-pc-platforms-the-snapdragon-x-series Quote Introducing an all-new naming convention for our next-generation of intelligent PC platforms: The Snapdragon X Series OCT 11, 2023 Qualcomm Technologies has been pushing the boundaries of innovation in powering consumer and commercial PC experiences since the introduction of the Snapdragon 8cx compute platform. We pioneered connectivity in computing with the world’s first 5G-enabled PC, introduced the world’s first dedicated neural processing unit (NPU) for artificial intelligence (AI) processing in a laptop, and we continue to lead performance per watt, enabling up to multiple days of battery life on a single charge. Today, I’m excited to debut an all-new naming architecture for our platforms to align with how we are revolutionizing the next generation of PC experiences: the Snapdragon X Series. https://www.nxp.com/company/blog/introducing-the-nxp-eiq-neutron-neural-processing-unit-npu:BL-INTRODUCING-THE-NXP-EIQ-NPU Quote Introducing the NXP eIQ® Neutron Neural Processing Unit (NPU) January 4, 2023 https://blog.st.com/stm32n6/ Quote STM32N6: Get a sneak peek at the future of AI-powered MCUs April 26, 2023 erh, it is getting pretty crowded? and that one can likely be assured none of the offerings are the same in anyway. if one has a board and wanted to try those NPU stuff, one is most welcomed to go ahead and try and perhaps tell the world what you achieved. there would be many keen audiences who would like to look at what you did and at least 'wow' about it. And by the way all that 'neural' stuff can mostly run on the *CPU*, and that there are plenty of accelerators for that AVX2, AVX512, SSE2, NEON, SIMD, GPGPU etc and that *GPUs* (Nvidia) has been around and has been the *industrial workhorse* for 'AI' stuff. there is also some thoughts about the NPU cacophony as above, while one start developing say on an NPU, no sooner than one gets started, another 10 competitors released another 20 NPUs and AI accelerators for your to try, and even then while you gets started, your SOC company release a new chip and a *brand new* NPU, obseleting your 'old' NPU while you are just about to 'get started' with you NPU stuff.
Guest Posted January 21 Posted January 21 28 minutes ago, ag123 said: there is also some thoughts about the NPU cacophony as above, while one start developing say on an NPU, no sooner than one gets started, another 10 competitors released another 20 NPUs and AI accelerators for your to try, and even then while you gets started, your SOC company release a new chip and a *brand new* NPU, obseleting your 'old' NPU while you are just about to 'get started' with you NPU stuff. We don't understand each other. I'm just an ordinary person. But, comrades, judging by the messages above, I think they consider themselves kings of the world only because they were able to mold Armbian out of shit. but you are right about one thing: in the field of software, you will always be late. The only way is for a strong AI to replace you soon. he will do what you cannot.
Gunjan Gupta Posted January 21 Posted January 21 3 minutes ago, Maksimarm said: We don't understand each other. I'm just an ordinary person. If you are not able to understand what ag123 posted, I am guessing you also have no Idea about what an NPU is. You just took some tech jargons, started a thread to look cool with no Idea about what you are asking for. And when people tried to ask for explanation or tried explaining to you that you are not making sense, then you simply started insulting everyone. If that is the case, I suggest we lock and delete this thread. Its not helping anyone.
c0rnelius Posted January 21 Posted January 21 7 minutes ago, Gunjan Gupta said: If that is the case, I suggest we lock and delete this thread. Its not helping anyone. Fun read though.
Recommended Posts