Pic8 reached 6 uploads today! | Poal: Say what you want.

[–] • 1 pt

Is it the reason why it's been acting funny and gay for the past 24 hours?

You might wanna look at the hosts, one of them is messing with the thumbnail and title code.

https://poal.co/s/Funny/403360

https://poal.co/domain/pic8.co/2

link

[–] • 3 pts

Thanks, been looking into it since anticlutch reported it to me.

It's an intermittent issue (lovely). Still running down the cause

parent
link

[–] • 2 pts

(edited )

I doubt it is related, but within the last couple days after uploading a file onto catbox, the button to click to copy the upload link now includes a space at the start before the https of the link it copies. Could that be breaking something?

I have no idea of the api logistics with pic8 and catbox, just thought it might be related. When I paste the copied link into IDM or the browser address bar I need to manually remove the space as it breaks the link.

Perhaps that could explain the intermittent nature. I'm not sure if pic8 mirrors the upload to one host and then mirrors that file on that host to another host, or if it is a single upload to pic8 that is mirrored to the others. If the first host is random and catbox is the first host, if that link is then mirrored to another host by pic8 then it might be a broken link being mirrored to the others.

Just theorizing. I know nothing of details, but I like to try and help with solving curious issues.

parent
link

[–] • 1 pt

I doubt it is related, but within the last couple days after uploading a file onto catbox, the button to click to copy the upload link now includes a space at the start before the https of the link it copies. Could that be breaking something?

I do believe this is the issue, actually. I can't tell if catbox changed something on their end (have to suspect this) because nothing changed on pic8's end. One of the remediations put in place today is to strip the (2) newline characters added to the catbox links.

Thanks for the suggestion

parent
link

Load more (2 replies)

[–] • 1 pt

/s/Anticlutchisgay

parent
link

[–] • 1 pt

You can tail the logs with a grep (to filter anything but errors)

parent
link

[+] [deleted] • 1 pt

[–] • 3 pts

(edited )

wrong encoding

Proper encoding (python)

>>> import binascii >>> val = int('110110', 2) >>> binascii.unhexlify('%x' % val) b'6'

parent
link

[–] [deleted] • 0 pt

proper.... python

Does not compute.

parent
link

[–] • 1 pt

python is in its own bracket

parent
link

Load more (1 reply)

[–] [deleted] • 1 pt

I would seriously look at how many files are in each directory on the pic8 server. I just read a line in a page I was reading on a search I did, the person said and this made something in me say this is the correct reason shit happens.

Even though the FS may be able to handle many files, the shell can't. This is wise. Just because you can store the files and retrieve them doesn't mean the system controlling them is not overworked. Likely not a database but a cpu being overloaded searching the database and the ram being constantly full from all the indexes.

I personally would add another drive to the system and then ls the files with dates older than a month or so and put them in the other drive or the slowest one for archiving in batches of 10 - 32K images even the thumbs limited. Heck it's not the size of the files but the number of them per directory that is likely fucking it all up. Even though supposedly limitless the ext4 fs does have to search inodes and index updating constantly.

I imagine those indexes are huge if the directories are not limited to a certain number of files and have the directories and subdirectories named alphanumerically so the system knows that image fucktheworld is in F maybe subdirectory s-w so it's like jump jump search index of that subfolder with 9k images and retrieve wanted files in 12 miliseconds flat not 200 ms since the index in each folder is short as fuck.

Then the system can find the folder effortlessly if to many files are in the bottom folder say the s-w one then have s-t then divide to s,t,u,v,w then even s"whatever symbol sorts first" - sla, so forth and on and on. First folder checked is the one chronologically most recent like a cpu's cache sorts what is kept and what is dumped.

If picman replaced a major drive with a new huge one and put to many files on that new drive then he'd be better of with slower multiple drives if his motherboard cpu setup can handle it then running 5 slower drives for retriving small files like thumbs and images would be far faster than on super fast sata that is limited to the fact that it's using one cable to talk back and forth where the 5 drives are using 5 cables so their combined throughput would be greater and smoother overall since it's not a mob of files but a instead a steady smooth flow of files orderly and then only the motherboard limitations or the networking will be the bottleneck or even PSU if it's running excessive HW for it's original intent.

I would seriously look at how many files are in each directory on the pic8 server. I just read a line in a page I was reading on a search I did, the person said and this made something in me say this is the correct reason shit happens. Even though the FS may be able to handle many files, the shell can't. This is wise. Just because you can store the files and retrieve them doesn't mean the system controlling them is not overworked. Likely not a database but a cpu being overloaded searching the database and the ram being constantly full from all the indexes. I personally would add another drive to the system and then ls the files with dates older than a month or so and put them in the other drive or the slowest one for archiving in batches of 10 - 32K images even the thumbs limited. Heck it's not the size of the files but the number of them per directory that is likely fucking it all up. Even though supposedly limitless the ext4 fs does have to search inodes and index updating constantly. I imagine those indexes are huge if the directories are not limited to a certain number of files and have the directories and subdirectories named alphanumerically so the system knows that image fucktheworld is in F maybe subdirectory s-w so it's like jump jump search index of that subfolder with 9k images and retrieve wanted files in 12 miliseconds flat not 200 ms since the index in each folder is short as fuck. Then the system can find the folder effortlessly if to many files are in the bottom folder say the s-w one then have s-t then divide to s,t,u,v,w then even s"whatever symbol sorts first" - sla, so forth and on and on. First folder checked is the one chronologically most recent like a cpu's cache sorts what is kept and what is dumped. If picman replaced a major drive with a new huge one and put to many files on that new drive then he'd be better of with slower multiple drives if his motherboard cpu setup can handle it then running 5 slower drives for retriving small files like thumbs and images would be far faster than on super fast sata that is limited to the fact that it's using one cable to talk back and forth where the 5 drives are using 5 cables so their combined throughput would be greater and smoother overall since it's not a mob of files but a instead a steady smooth flow of files orderly and then only the motherboard limitations or the networking will be the bottleneck or even PSU if it's running excessive HW for it's original intent.

link

[–] • 2 pts

There is very little interaction with the file system.

Files are only stored on the FS during initial upload. After that, everything is a simple string value in a database. Lookups are basic string comparisons, very little computational effort, even for million of entries (only at 100k)

I do apprecaite the thought. I believe the recent issues with images sometimes not being found is an operation issue on pic8's side, along with some fuckery with an underlying host. Been working on fixing some of these operational issues, but sadly the codebase was written back when I was a much less experienced developer.