-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About jpeg and future development of NFT marker creator #6
Comments
To generate the marker files is needed some information about the image:
The function "readImageFromFile" gets all the information to set a struct and then other functions can generate the marker files. Using an HTML input gives only the Uint8 array, the width, and height. I tried setting a placeholder DPI, but it didn't work. I was trying to generate a new jpeg file and store it in the MEMFS and then read it "locally" with the GenTexData.c, so I wouldn't need to change the original code from the unity artoolkit. Maybe the solution is easy, but I didn't have much time to work on it 😞 |
Thank you @Carnaux i will look into it when i have a bit of time. |
@ThorstenBux Can you help me understand better the marker generation? |
I can try :). What are your questions? |
I solved most of my questions working today 😃, I'm generating the iset and fset files, but they don't work, they have drastically different file size than the generated by the genTexData.exe. |
well, that's a great progress :) good luck, as soon as I will have time I will help you |
That really strange @Carnaux but as said Nicolo a great progress! |
Just to document one of my attempts: I tried to execute the genTexData.exe directly in node.js. If you don't pass a file as parameter the expected error message appears, so it runs and does "console.log", but when I pass the path for the file it won't show any sign that it is running. I think that the .exe is waiting for a DPI input and is getting the code stuck. I tested with a simple hello world .exe compiled from c++ that just outputs and the code worked, when I added an input it got stuck. A possible solution is the use of child_process.spawn(). If the input problem gets solved, the data got by EXIF.js can create default values for the inputs of the .exe. What the current code can do:
|
@Carnaux I think that if you can come up with specific questions and maybe an online example to try (not mandatory) you can ask to Thorsten. He's very will to help. Just come up with some questions here and I will see to make it happen. |
Update: I posted about an error dpi_num, but it was a typo error, and it was solved. I think I figured out what was the deal. The way I was passing the pixel data as a parameter was wrong, solving that made the generation almost equal to the genTexData.exe, all logs are being shown, and all generation steps are seen to be working. But I noticed that the files still different sizes so they don't work with the NFT example, I believe it is the ARUint8 pixel array data structure, the feature extraction doesn't generate the correct values. @ThorstenBux, I saw in a forum that the ARUint8 pixel structure changes depending on the SO, it apply in this case too? PS: Now all the steps of the marker generation work, but it takes 25 minutes to finish. The repository was updated, so anyone can test the generation times. |
I will look at for the ARUint8 issue. |
I developed a node app too, that uses the same markerCreator.min.js as the browser version(the current one), just to test generation times. The results:genTexData.exe: 3 min Some differences:genTexData.exe:Node app:The values of filtered features are all slightly higher, but it doesn't log feature coordinates for high DPI values, only for 75 or below. Currently, it is faster than the browser, but I don't know how it will behave when it is hosted. Browser:The extracted and filtered values aren't correct, but it logs all coordinates for all DPIs. |
Having wrong/few features could be the cause for the non-recognisation on Web. |
I think that our 'first' aim was to make it work with Web technologies, but maybe this computation is too heavy for the browser. I think we have to focus on make it work on node environment, and then, eventually in the future, port it also on the browsers. But for a lot (maybe every) use case, to generate the descriptors 'offline', and then load them on the web, will be enough. |
If you agree with me, you can focus your efforts on experiments/debug with node version only, like you did on your last post |
@Carnaux yes maybe a node version will be also fine and easier to develop. @nicolocarpignoli i can make my own Nft markers also under linux, but you need to build the artoolkitx project and this can be a little cumbersome to insall all the necessary dependencies, so in the scenario of a "normal" user it's a kind of blocking thing, better a lot an alternative app that can run on every OS... |
Yeah, I agree. I'm building a new node app, and when it is finished I will reorganize the repository with the main app and other branch to the NftMarkerCreator.min.js build. |
Done, the app is on the master branch, you guys can test it. But I still stuck at the feature extraction, I tested changing the order of the RGB pixel array, but still the same as I said, the node app only logs feature coordinates for low dpi values. |
Finally!!!! With this last commit, the code works, you now can generate custom NFT Markers!!!! 🎉 Please, test and report any errors or bugs! @kalwalt @nicolocarpignoli |
I've tested and it is running with other jpg files as well. but I can't see any output files .fset .iset that are created. I've deleted the existing ones added a new jpeg to input. It runs with this output
But does not create output files |
The process finishes? On my PC it took around 14 min to generate. |
It looks like it gets stuck because of this |
Weird, that log is in a function that should not be called. Can you send me the image that you are using? |
Edited the jpeg now I get |
Sorry that log is out of the c version of the tool :) |
Image on the way |
(I hope google doesn't change the file) |
Ok, got it now. The link above: That image fails for some reason I don't understand. |
It is the jpeg decoder, I will add this to a to-do list for a future update. Edit: @ThorstenBux https://images.app.goo.gl/6JFAu3reU3bjzUKP6 this one works. I added the handling for empty EXIF info. Edit2: It was generated in 5s!! 😮 |
One thing I realized: The images I was using for testing were all 1600 < in width and height, so they were big images. Small images are much faster to generate, but I believe that it has few details for the feature extraction. |
@Carnaux @ThorstenBux I will test now! |
🎉 Yesss, it works !! 🎉 see the log |
Tested with this my image https://www.kalwaltart.com/assets/images/uploads/cubist_dragon.jpg i used for my AR made with Artivive read this article |
Hi! I tried on macbook pro. It finishes in ~10-15seconds but no descriptors generated.
any idea? |
Tried with thorsten image above edit: following the readme, I was not able to do this point
Cannot find that line, maybe Readme is obsolete? I'm doing this on dev branch. |
which version of emscripten do you have guys? |
I always got those warnings about libjpeg, but it never failed the build 😕 Yes, it is outdated, sorry, I will change it, is just in case your libjpeg folder is in other place. My version is 1.39.4 |
the build does not failed actually. I see that the 'output' directory is not present. I have manually created it but does not work either |
The dev branch just generates the NftMarkerCreator.min.js, to generate the marker itself you have to use the master |
Yes, I was just to write it! I didn't understand that! |
So the complete flux, as for now, should be:
I got error with this flow.
Also, I was able to create .iset files using the Editor.html from dev branch but they did not seem to work. |
That flow is just if you want to make some changes to the min.js, the master branch is ready to use. I'm making some changes in the dev branch, making it cleaner, the html editor is discontinued. |
I made the changes in the dev branch, it is cleaner now. One thing I've been thinking is to merge the .iset, .fset and .fset3 files in just one maybe is more practical? What do you guys think? @kalwalt @ThorstenBux @nicolocarpignoli |
Hi I tried with the pinball.jpg and it's come out with this |
You type "y" and press enter after that will have other dialogs, for Width and Height of the image, and for DPI, if you want to inform one(or just press enter, the code will use 72 as default) |
Thank you It's working ,get the files |
it works on master branch! Great!! As next step I would suggest to automate it even more:
For each of them, if can't retrieve, fallback to a default. I don't know if this is all possible but should be great. Anyway, your idea to create a basic HTML page as generator was great. Something very similar to create NFT images will do, like:
What do you think? |
Next weeks I will be a bit busy to try to port NFT jsartoolkit5 on AR.js but I will always provide support for this, and when I will be more free even 'development' support :) |
I think a node app is the most reasonable solution at the moment, much easier than having to clone artoolkitx, install all the dependencies (what a panic!) and compile the C++ project . Above all if you are not an experienced C++ developer this is discouraging... |
@nicolocarpignoli the node app already does the automation, it only asks to input info if the decoder can't get the EXIF info. But the c++ code has some advanced inputs, e.g the precision of the feature extraction, I will add those options later. I just uploaded a simple page with the browser version: https://carnaux.github.io/NFT-Marker-Creator-Web/ It is not faster, but for small images is more practical. |
@Carnaux i tried with the browser version, yes it seems it takes longer to generate the data set... |
I just added PNG support for the node app!! It works with grayscale and RGB/RGBA images. Edit: Also added to the browser version! |
Great @Carnaux !! Thank you! |
@Carnaux does it make sense to merge and close this PR and work directly on specific features, that solves specific issue? |
Yes, this was just for general conversation. |
We continue here the discussion started on my PR for NFT kalwalt/jsartoolkit5#2 (comment)
@Carnaux we can leave here our thoughts and solutions for the problem.
The text was updated successfully, but these errors were encountered: