Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.misc > #37398
| Subject | Re: Contents of file changed while application is reading it |
|---|---|
| Newsgroups | comp.os.linux.misc |
| References | (13 earlier) <ttpr01$279gm$50@dont-email.me> <dYudnfNen9mG55z5nZ2dnZfqn_ednZ2d@earthlink.com> <ttsejp$279gm$64@dont-email.me> <qPadnQSthu_CfJ_5nZ2dnZfqn_ednZ2d@earthlink.com> <ttuppo$279gm$74@dont-email.me> |
| From | "28A.I873" <28A.I873@noabzba.net> |
| Organization | mitotic resell |
| Date | 2023-03-04 03:32 -0500 |
| Message-ID | <S7icnftS38qWn575nZ2dnZfqn_SdnZ2d@earthlink.com> (permalink) |
On 3/4/23 1:51 AM, The Natural Philosopher wrote:
> On 04/03/2023 06:12, 28A.I873 wrote:
>> On 3/3/23 4:28 AM, The Natural Philosopher wrote:
>
>>> And I also discovered the PHP is utter crap with large data sets. My
>>> PHP now calls a C program to access the database and present the
>>> selected data as an output stream.
>>
>>
>> PHP *can* do almost anything. However HOW to do 'almost
>> anything' can be a long long search.
>>
> Oh it worked fine with a smaller data set, but the same query on a very
> large data set simply didnt work.
Probably 8/16-bit reasoning at work ... short strings,
or recs of short strings - with INT indexing.
Theoretically it's Getting Better.
Ran into a problem with Free Pascal ... needed to
do 64-bit seeks on gigantic files. The DOC said
that their seek()s wer 64-bit - but in reality
they were 32-bit. The same segments would come
up over and over, every time you crossed the
old 32-bit boundaries. Had to write a 'C' helper,
and even that required some compiler flags to
force 64 bits.
> I guess some internal buffer limit was exceeded
Yep. Gotta break things up into SMALL bits if
you want it to work. Kludgy.
>>> In the context of concurrency, I also ran into a weird PHP buffering
>>> issue. . In that an uploaded file to a php equipped web server is
>>> available as a temporary entity, which must be converted to an
>>> explicit file if you want to keep it or access it. Which I duly did
>>> before invoking an SQL command to 'load data file' . Which always
>>> failed to work. Because PHP didn't actually write the file until it
>>> exited.
>>>
>>> I cannot remember what sketchy sort of kludge I eventually used., It
>>> may well have been a copy file command that finally resulted in it
>>> being written to the OS where the SQL daemon could read it.
>>
>> Always a few irregularities that don't fit the paradigm.
>> However if they're FEW you can manually tweak 'em.
>>
> One spends a lot of time working round language bugs.
Gee ... you noticed !!! :-)
>> John O' Groats ... isn't that the bitter end of the UK
>> mainland ? I've toured it on Earth ... seems like an almost
>> ideal place (to ME) to build a little retirement cottage :-)
>>
>> But UK taxes are too high and all the pols these days seem
>> to be out of a Python sketch .........
>>
>> I had some HUGE old databases - some in PICK-ish MV format
>> and others flat-file ... that I needed to preserve for
>> posterity. 10+ years of detailed field observations in
>> each. Alas it WAS possible to over-ride the official
>> choice of responses, and some managed to put stuff from
>> one category into another. Wound up with about 300
>> "anomalies". Fortunately MOST weren't TOO awful - so
>> I fixed them in MySQL by hand. Do check out "DBeaver"
>> interface ... simple but effective.
>>
>> Ah ... an interesting technical question about 'C' ...
>> assume you're looking for the position of a given
>> character in a string .......
>>
>> for(x=0;x<strlen(TheStr);x++)
>> {
>> if(TheStr[x]==TheChar) return(x);
>> }
>> return(-1);
>>
>> is a pretty common approach. HOWEVER, the QUESTION,
>> is "strlen(TheStr)" calculated over and over again
>> for every iteration ??? That seems SLOW, esp if
>> TheStr is BIG.
>>
> Not with a modern optimising compiler I think
>
>> Of late I've been using :
>>
>> for(x=0;TheStr[x] != 0;x++)
>> {
>> if(TheStr[x]==TheChar) return(x);
>> }
>> return(-1);
>>
>> instead of using strlen().
>>
> I think that what you will find is that strlen(TheStr) is stored in a
> register and never re-evaluated...compilers are pretty good at finding
> operations in loops whose answers do not change.
Well, that's the Big Question - is it calculated ONCE or
EVERY iteration. The SAFE approach is Every Iteration,
Just In Case.
Gonna have to do some benchmarks with GCC & Clang ....
Anyhow, I've writ three 'utilities' in the last two
weeks in 'C' ... so these questions come to mind.
>> Works just as well, at least in fairly simple cases,
>> but seems much faster than running strlen() over and
>> over a zillion times.
>>
>> I can envision a few circumstances where TheStr gets
>> seriously mutated and strlen() is probably best to
>> deal with its new length - but for these straight-up
>> sorts of things ....
>>
>> And no, I tend to avoid pointer solutions because they
>> are not nearly so clear (and often have MANY more
>> little sub-steps). The compiler turns the ordinary
>> index numbers INTO pointers/pointer-math anyhow ... so
>> may as well be CLEAR & OBVIOUS.
>>
> I use pointers a lot. I never have a problem understanding it. I can
> 'think in pointers' these days.
Now go back to a pgm you wrote 10 years ago ..... :-)
OR, worse, if the New Guy has to look at your
10-year-old code. (horrible Truth - they're
probably not as good/wise as you are - will
probably try to re-write it in Python or
whatever the 'popular' language of the week
may be).
MY take is that the compiler does, always DID, turn
those easily-comprehensible indexes into pointers
and pointer-math anyway. So, you rarely GAIN anything
by writing an all-pointer approach. These days
compile-time is usually SECONDS too ....
Just For Fun you can write a few benchmarks ... I think
you'll see I'm usually right here.
> The one that blew me up - not sure whether it was the primitive compiler
> or my misunderstanding the syntax - was an array of pointers to
> functions. So I could call different functions depending on a value in a
> variable.
> I.e the equivalent of
> Case(x)
> 1: f1(p); break;
> 2:f2(p); break;
> ....
AAAuuuughhh !!!
Yea, I *have* done that. There ARE a few places it's
The Best approach. BUT, it's SO easy to screw it up !
It's usually NOT the compiler ......
Anyway, note my little crusade against "complex"
coding. "Clear & Obvious" beats 'em all.
>> Came across a web page the other day about 'strstr()'.
>> The first advice code was about three lines long. Then
>> some other hotshot said "Oh ! You should be using
>> POINTERS !" and offered the equiv - about *30* lines
>> long and almost incomprehensible at first glance :-)
>>
>> Hey, I need code *I* can understand one, five, ten,
>> years down the line !
>
> And that is why modern compilers are so good. You can write the code in
> an understandable way, and the compiler will turn it into uber fast
> greased incomprehsenible weaselshit
Yep. Compiler-writing is damned damned GOOD these days.
> Last time I dissassembled some C compiler output I was impressed - it
> was better assembler than I would have written!
I've seen that - even for micro-controllers ! I've always
used hand-writ assembler for those ... but the latest 'C'
compilers use trix I'd never thought of to slim it down.
Impressive !
> The days of the compiler slavishly following the source code are long gone.
They've got smart.
Which, 99.999% of the time, is a good thing .... :-)
Aw shit - it's after 3AM, again .......... lucky
tomorrow is Saturday ...........
Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar
Contents of file changed while application is reading it Clark Smith <noaddress@nowhere.net> - 2023-02-26 18:52 +0000
Re: Contents of file changed while application is reading it Rich <rich@example.invalid> - 2023-02-26 19:20 +0000
Re: Contents of file changed while application is reading it Robert Heller <heller@deepsoft.com> - 2023-02-26 21:45 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-26 23:05 +0100
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-26 23:38 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-26 23:32 +0000
Re: Contents of file changed while application is reading it "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2023-02-26 19:06 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-27 00:26 +0000
Re: Contents of file changed while application is reading it "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2023-02-26 20:48 -0500
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-02-27 09:03 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-27 09:58 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 11:16 +0100
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-02-27 15:20 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 10:34 +0100
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-02-27 09:49 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 11:13 +0100
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-27 10:02 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 11:13 +0100
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-27 10:17 +0000
Re: Contents of file changed while application is reading it "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2023-02-27 12:27 -0500
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 19:19 +0100
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-02-27 21:46 -0500
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-28 13:33 +0100
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-02-28 10:49 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-28 16:21 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-01 01:05 -0500
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-01 21:57 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-01 22:53 -0500
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-02 19:06 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-02 23:08 -0500
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-03 05:23 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-03 09:36 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-03-03 11:19 +0100
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-03 15:58 +0000
Re: Contents of file changed while application is reading it "Carlos E. R." <robin_listas@es.invalid> - 2023-03-03 18:47 +0100
Re: Contents of file changed while application is reading it Rich <rich@example.invalid> - 2023-03-04 16:24 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-03-04 18:40 +0100
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-03 18:44 +0000
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-04 01:25 +0000
Re: Contents of file changed while application is reading it Rich <rich@example.invalid> - 2023-03-04 16:21 +0000
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-04 23:55 -0500
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-05 05:35 +0000
Re: Contents of file changed while application is reading it Allodoxaphobia <trepidation@example.net> - 2023-03-05 13:48 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-05 14:01 +0000
Re: Contents of file changed while application is reading it Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2023-03-05 19:16 +0000
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-06 00:43 -0500
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-06 00:42 -0500
Re: Contents of file changed while application is reading it "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2023-02-28 12:01 -0500
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-01 01:17 -0500
Re: Contents of file changed while application is reading it Robert Riches <spamtrap42@jacob21819.net> - 2023-02-28 04:13 +0000
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-28 13:16 +0100
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-01 01:32 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-01 10:30 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-01 22:48 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-02 09:41 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-02 23:39 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-03 09:28 +0000
Re: Contents of file changed while application is reading it "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2023-03-03 12:16 -0500
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-04 01:36 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-04 07:15 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-04 02:47 -0500
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-04 01:12 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-04 06:51 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-03-04 03:32 -0500
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-03-04 09:54 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-04 10:09 +0000
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-03-04 11:22 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-04 11:58 +0000
Re: Contents of file changed while application is reading it Richard Kettlewell <invalid@invalid.invalid> - 2023-03-05 16:43 +0000
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-06 09:20 +0000
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-04 20:24 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-05 13:40 +0000
Re: Contents of file changed while application is reading it Dan Espen <dan1espen@gmail.com> - 2023-03-05 11:29 -0500
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-06 01:21 -0500
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-06 01:06 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-03-06 09:28 +0000
Re: Contents of file changed while application is reading it "28B.I874" <28B.I874@noabzba.net> - 2023-03-06 08:54 -0500
Re: Contents of file changed while application is reading it Dan Espen <dan1espen@gmail.com> - 2023-02-26 17:35 -0500
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-26 23:39 +0000
Re: Contents of file changed while application is reading it "28A.I873" <28A.I873@noabzba.net> - 2023-02-26 22:42 -0500
Re: Contents of file changed while application is reading it "Carlos E.R." <robin_listas@es.invalid> - 2023-02-27 10:45 +0100
Re: Contents of file changed while application is reading it The Natural Philosopher <tnp@invalid.invalid> - 2023-02-26 23:31 +0000
csiph-web