mozilla :: #pdfjs

17 Jul 2017
14:10mukulyury: hi, I am trying to compare data `chunks` in chunked_stream file with testPdfData.js(size ~3.5MB) file, as you suggested earlier. But unfortunately viewer is not loading, I think because of large size of testPdfData.js file. Can you help me with this?
18:16mukulyury: Hi
18:16yuryhi
18:17yuryyou can add to worker_loader bypassing system.js
18:17yuryuse importScritps
18:18yury(or <script> for viewer.html)
18:21mukulso testPdfData.js should contain pdf data(new Uint8Array([....])) or node script that generate that data
18:22mukulsomething like `new Uint8Array(fs.readFileSync())` ?
18:27yurymukul: what `&#39;new Uint8Array(&#39; + fs.readFileSync(....)` prints?
18:29mukulIt will give `Uint8Array([....])` with huge PDF data inside typed array.
18:31yuryso change it somehow to produce `self.testPdfData = new Uint8Array([....]);` as one big string and save it in the JS file
18:32mukulWe can run this code only inside node environment not in browser, that is why I am confused and trying to generate raw data with node script and then save it in a file in src/core folder
18:32yuryI don&#39;t understand
18:33yuryyou need to generate JS that will run in the browser
18:33yurynode will generate this JS
18:34yuryI don&#39;t know why browser cannot run code in `self.testPdfData = new Uint8Array([....]);` form -- it looks valid browser JS file to me
18:34mukulI am generating raw pdf data using node script, and then saving it in a file(testPdfData.js) something like `exports.testPdfData = new Unit8Array([....])`.
18:35yuryis there any error?
18:35mukulYou are right, we can run `self.testPdfData = new Uint8Array([....]);` in browser. But we cannot run `fs.readFileSync`
18:36yurywhy do we need to run fs.readFileSync?
18:36mukulNope, no error. But viewer is not loading, seems like due to large file size.
18:36yuryokay, to bypass system.hs
18:36yurysystem.js
18:36yuryuse importScripts() in the worker_loader
18:36yuryand use self.testPdfData
18:40mukulokay, now it is loading, bit slow though maybe due to lots of comparisons.
18:43yurycomparisons must be fast
18:44yuryis it just simple loop
18:45mukulyes
18:45mukuleh, it hanged my computer.
18:49duziyury: From what I observe, resizeRgbImage(...) skips some pixel data while resizing. Is it ok?
18:49yuryduzi: maybe it&#39;s okay -- it depends on users
18:50duziyury: Also, when would it execute. Can you give a real-life example form pdfjs&#39; perspective?
18:51yuryduzi: I don&#39;t remember from top of my head, it&#39;s probably needed when image is trying to be matched to mask
18:52duziAlright. dimensions of mask >= dimensions of image, right?
18:52yury(they can be different) so in this case, resize will increase size
18:52yuryboth ways
18:53yurymax(width) and max(height)
18:53mukulyury: We have to apply some other looping technique? I am comparing using a simple for loop.
18:54yurywhat if you comment comparision, is it fast?
18:54yuryand loop
18:55mukulyes, it is fast when comparisons are commented out.
18:56mukulOkay, I think I am console.log(&#39;.&#39;) inside comparisons, that is why it is slow.
18:56mukulremoving it makes it fast
18:57duziyury: Looking at https://github.com/mozilla/pdf.js/blob/master/src/core/image.js#L264-L274 seems resize will only increase size. I don
18:58yurysure
18:58yurymukul: are you comparing only items that .set() is about to set?
18:59mukulYes, in both onReceiveProgressiveData, and onReceiveData
18:59yurycool
19:00yuryso you found the issue already?
19:02duziyury: Also, have a look at https://github.com/mozilla/pdf.js/blob/master/src/core/image.js#L576-L578 where drawHeight and drawWidth are used. I can&#39;t find any usage where image will be downsized. Is it so?
19:04mukulyury: I think we are getting wrong data in onReceiveProgressiveData
19:12yuryduzi: sounds like truth
19:12yuryso testing can to be done for increasing e.g. 2x2 -> 3x3 or 4x4
19:13duziYeah
19:14yurymukul: cool add more logging and comparison along the way of data path
19:14yuryincluding main thread
19:15yuryyou can track onReceiveProgressiveData position via some global vars
19:15mukulyury: It is very rare case, I am getting this error only when I load the viewer first time.
19:16mukulMaybe it is caching?
19:17yuryyes, there is a way to refresh page without cache, I think via holding ctrl key
19:17yuryor shift
19:18yuryhttps://support.mozilla.org/en-US/questions/1103414
19:20mukulHmm..., but still loading just fine.
19:21yurymaybe worker loading delay
19:38mukulyury: how to fix this ^^^^? Any idea, because I can&#39;t reproduce the error condition(even with disabling cache and with complete reload).
19:39yurythere is external/systemjs/plugin-babel-cached.js
19:39yurythere is line `return loadCache(load.address, hashCode);`
19:41yuryif you return null for some file(s) e.g. /shared\/util.js/.test() based on address
19:41yurymukul: it will kinda of skip cache
19:42yurymukul: if it&#39;s too complex, just set &#39;false&#39; for isCachingPossible in system.config.js
19:46mukulLooks like it is not a problem of caching, still loads correctly. Bad luck.
19:47mukulsetting isCachingPossible = false, did not help
19:49yurymukul: did you disable browser caching?
19:50mukulyes.
19:52yury
19:55yurymukul: probably intermitent, just try to find a way to increase probabilty of reproducing the failure
19:56yurysometimes it&#39;s just like that
19:57mukulYeah, I have to wait for a while and not load the viewer. That&#39;s how I am reproducing this issue currently.
19:58yury(means that isCachingPossible = false shall work)
19:58yuryif you wait
19:59mukulnot sure why it is not working.
19:59yurysee https://github.com/mozilla/pdf.js/blob/master/external/systemjs/plugin-babel-cached.js#L17 , means 60 min wait
20:53duziyury: fillRgb(...) in turn calls getRgbBuffer(...) and resizeRbgImage(...). Should I write tests for those functions too Or only the tests for fillRgb(...) would suffice?
21:23yuryduzi: only fillRgb() will work just fine
21:23duziAlright
18 Jul 2017
No messages
   
Last message: 10 days and 1 hour ago