Author Topic: Info field string length limitation  (Read 13749 times)

0 Members and 1 Guest are viewing this topic.

Padanges

  • Newbie
  • *
  • Posts: 179
Info field string length limitation
« on: September 27, 2016, 10:06:08 AM »
Hi,
after saving a string with more than 65436 chars into a DB Metadata Info Field you can get PDFe DB to hang-up while scanning the file which the changes were made to. In such situation you won't be able to Stop the scanning (it won't respond), and PDFe will hang up. You will have to kill the process. Only entirely removing the document from the DB will help you to get PDFe to complete initial Disk scan.
I really recommend to update CommitChanges() method with string length verification and auto-truncation (perhaps with some I/O Message?). Adobe Acrobat has no problems opening this extra long "info field" string.


Regards

P.S. What are possible situations where CommitChanges() fails? Can the method return a 'fail' status? What's use of such code?:
Code: [Select]
if (FileMetadata.CommitChanges()) { pdfe.echo(Filename + ' - OK');}

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #1 on: September 28, 2016, 03:56:43 PM »
I'm not getting that DB problem with the long string, but yes the length must be checked and trimmed to an acceptable value.

Regarding the CommitChanges fail situation. The fail can be related to many things, like file in use, unhandled situation exception (unknown error), etc. A status code is preferable, and may be provided in a future implementation, but knowing that it failed is sufficient to be aware that a file in particular needs to be checked.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #2 on: September 29, 2016, 09:13:35 AM »
Quote
I'm not getting that DB problem with the long string, but yes the length must be checked and trimmed to an acceptable value.
Are you suggesting we should always check that in our scripts before commiting changes?

Quote
Regarding the CommitChanges fail situation. The fail can be related to many things, like file in use, unhandled situation exception (unknown error), etc. A status code is preferable, and may be provided in a future implementation, but knowing that it failed is sufficient to be aware that a file in particular needs to be checked.
Can failed file reading (or any other situation) pervent us from using "DB Only" option for saving changes with: CommitChanges(false) ? Correct me if I'm wrong but CommitChanges(false) stands for "DB only" option for changes.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #3 on: September 30, 2016, 07:58:23 AM »
Tooltips could also limit string length relative to the main Window size. I have had situations (messing around with LONG_PATH file names and Custom Field Info char limit) where the string would not fit into a tooltip, and the tooltip used to take whole screen width, even without showing that long string at all.
By the way, how can we get the PDFe current main window size in pixels?

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #4 on: September 30, 2016, 04:24:24 PM »
Quote
I'm not getting that DB problem with the long string, but yes the length must be checked and trimmed to an acceptable value.
Are you suggesting we should always check that in our scripts before commiting changes?
And why not? But no, the trim will be done internally.

Quote
Can failed file reading (or any other situation) pervent us from using "DB Only" option for saving changes with: CommitChanges(false) ? Correct me if I'm wrong but CommitChanges(false) stands for "DB only" option for changes.
No. As long a DB entry exists for the related file, the DB only metadata update will work just fine without checking for/touching the file itself.

By the way, how can we get the PDFe current main window size in pixels?
I'm not aware of any ready available way to do it from a Active Scripting script, at least without the help of a third-party COM control, or a script engine with the functionality built-in (i.e. I'm sure it can be done with the ActivePython engine).

You may use this trick to get the full screen resolution.
Code: [Select]
var objIE = new ActiveXObject("InternetExplorer.Application");
objIE.Visible = false;
objIE.fullscreen = true;
var width = objIE.width;
var height = objIE.height;
objIE.Quit();
pdfe.echo(width + "x" + height);

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #5 on: October 14, 2016, 06:56:21 AM »
Quote
And why not? But no, the trim will be done internally.
Sure we can do that, but it would be nice if you could confirm whether we should update our all CommitChanges procedures in the scripts or not.
Quote
You may use this trick to get the full screen resolution.
Thanks. I thought that settings like window size and position get saved in the registry by default.

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #6 on: October 15, 2016, 04:48:21 PM »
Quote
And why not? But no, the trim will be done internally.
Sure we can do that, but it would be nice if you could confirm whether we should update our all CommitChanges procedures in the scripts or not.
You will need to update your scripts only if you need to know that a specific field got trimmed because it tried to store a too long string. Allowed field length, for custom fields, is now 1000 characters.

Quote
Quote
You may use this trick to get the full screen resolution.
Thanks. I thought that settings like window size and position get saved in the registry by default.
Sure, but in the registry you have the previous running session values, not the current ones. These settings are obviously saved to the registry when the specific window close, or the program ends for the main window case.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #7 on: October 21, 2016, 01:04:17 PM »
Quote
You will need to update your scripts only if you need to know that a specific field got trimmed because it tried to store a too long string. Allowed field length, for custom fields, is now 1000 characters.

This is a sad thing. I've hit the limitation while simulating for possible cross reference, legacy tags etc string overflow. Perhaps you could leave a possibility to have a 65K char limitation (or even more?) as an optional check in Custom Fields Settings while creating (Ading) new Custom Field? I think there's a need to have a possibility to expand string length for specific info fields.

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #8 on: October 22, 2016, 03:59:32 PM »
The custom fields are implemented in a way that all (100 possible) share a common 65k maximum storage space, so if you put 65k in just one field, there will be no space left for any other. Having a common ground limit is desirable. Even because, storing such amount of data in the fields would affect badly the program's responsiveness.
The standard fields are limited to just 255 bytes and it's sufficient for common scenarios.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #9 on: November 22, 2016, 10:35:23 AM »
After your increased limitation implementation  I will need a new way to store cross-reference ID lists. Maybe you could advise how to implement a "longer string" saving to the db using PDFe?

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #10 on: November 22, 2016, 11:55:40 PM »
No idea. Depending on the utilization of that list, maybe you could attach it to the PDF?

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #11 on: December 01, 2016, 08:14:47 AM »
Quote
Allowed field length, for custom fields, is now 1000 characters.
Please consider allowing field length to be at least 4096 characters long. We used to have 65K and it had no visible impact on performance. I think this change is not substantial if still compared with 65K.

RTT

  • Administrator
  • *****
  • Posts: 907
Re: Info field string length limitation
« Reply #12 on: December 06, 2016, 11:11:04 PM »
I have no problems lifting this limit but you can be sure you will experience performance degradation, if you put all this data in a metadata field.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #13 on: December 16, 2016, 09:09:04 AM »
Quote
I have no problems lifting this limit but you can be sure you will experience performance degradation, if you put all this data in a metadata field.
Yes, please, then do so. I do understand the possible performance penalties but that's an unlikely situation for most users. I run into additional difficulties while experimenting with PDFe scripts. Currently I'm testing a way to extend Info Field memory by using external files so this is primary just to retain some freedom for experimentation :)
4K should be just fine. 1K is too restrictive, IMHO.

Padanges

  • Newbie
  • *
  • Posts: 179
Re: Info field string length limitation
« Reply #14 on: December 16, 2016, 11:13:57 AM »
In addition to this, please help to get the numbers straight:
Custom field count limit is 100.
Allowed custom field length is 4096 (previously: 1000) characters.
Every custom field for each entry shares a common 4k (previously: 1k) storage space.

Is that correct? I need to assign values for limit constants.