Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #8247 > unrolled thread

Looking for simple brute force solution

Back to article view | Back to comp.programming

Contents

  Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-03 23:18 +0000
    Re: Looking for simple brute force solution fir <profesor.fir@gmail.com> - 2018-01-03 16:49 -0800
      Re: Looking for simple brute force solution fir <profesor.fir@gmail.com> - 2018-01-03 17:11 -0800
        Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 10:14 +0000
          Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-04 10:52 -0500
            Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 16:46 +0000
              Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-04 12:37 -0500
    Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-03 21:28 -0500
      Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 09:42 +0000
        Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-04 10:16 -0500
          Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 17:26 +0000
            Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 21:19 +0000
    Re: Looking for simple brute force solution Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-01-04 02:29 +0000
      Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 09:55 +0000
        Re: Looking for simple brute force solution Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-01-04 14:29 +0000
          Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 16:55 +0000
            Re: Looking for simple brute force solution Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-01-04 20:36 +0000
        Re: Looking for simple brute force solution Robert Wessel <robertwessel2@yahoo.com> - 2018-01-04 12:03 -0600
          Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 21:28 +0000
            Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 21:50 +0000
              Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-04 23:06 +0000
              Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-04 21:35 -0500
                Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-05 05:59 +0000
                  Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-05 16:48 +0000
                  Re: Looking for simple brute force solution Richard Damon <Richard@Damon-Family.org> - 2018-01-06 08:50 -0500
                    Re: Looking for simple brute force solution root <NoEMail@home.org> - 2018-01-06 16:47 +0000

Page 2 of 2 — ← Prev page 1 [2]

#8267

root <NoEMail@home.org> wrote:
>>
>
> I was surprised to find that only one solution exists to the problem.
> I let the program run through all cases and it finished in a little
> over 4 minutes. It ran through 3,486,784,401 (3.5B) cases.
>
> I really don't like the results. I'll have to ponder that before
> continuing.



The data I showed was for 2016 BAC annual income. I repeated the
test for WFC and JPM and all the weightings came out different.
That means the relationship between total interest income and
operating income is from different combinations of the entries.
That can't be, so the data are wrong. This whole exercise seems
to be for nought. Damn. I'll come back after a while to verify
that I haven't made a mistake, but hand calculations confirm
the results of the program.

Thanks for everyone's help.

[toc] | [prev] | [next] | [standalone]

#8268

On 1/4/18 4:50 PM, root wrote:
> root <NoEMail@home.org> wrote:
>>
>> We'll see how the case currently running turns out. I only have to work
>> out the solution for four or five cases then use that solution to
>> unravel data for about 420 data sets.
>>
> 
> I was surprised to find that only one solution exists to the problem.
> I let the program run through all cases and it finished in a little
> over 4 minutes. It ran through 3,486,784,401 (3.5B) cases.
> 
> I really don't like the results. I'll have to ponder that before
> continuing.
> 

Due to the rounding issue, one may be a bit better than another, but you 
should be able to replace any running sum by the values that make it up, 
as they should be about equal, as that was part of the definition of 
being a running sum.

If that isn't happening, then you are just running into the issue of an 
under constrained fitting operation, you can get an answer, but it might 
not really mean anything.

[toc] | [prev] | [next] | [standalone]

#8269

Richard Damon <Richard@Damon-Family.org> wrote:
> On 1/4/18 4:50 PM, root wrote:
>> root <NoEMail@home.org> wrote:
>>>
>>> We'll see how the case currently running turns out. I only have to work
>>> out the solution for four or five cases then use that solution to
>>> unravel data for about 420 data sets.
>>>
>> 
>> I was surprised to find that only one solution exists to the problem.
>> I let the program run through all cases and it finished in a little
>> over 4 minutes. It ran through 3,486,784,401 (3.5B) cases.
>> 
>> I really don't like the results. I'll have to ponder that before
>> continuing.
>> 
>
> Due to the rounding issue, one may be a bit better than another, but you 
> should be able to replace any running sum by the values that make it up, 
> as they should be about equal, as that was part of the definition of 
> being a running sum.
>
> If that isn't happening, then you are just running into the issue of an 
> under constrained fitting operation, you can get an answer, but it might 
> not really mean anything.


The problem is with the data. The solution found is exact. The only way
that could happen if the data are intentionally  contrived to yield an
exact fit with the wrong entries to mask a correct fit with slightly
different numbers. Here are two solutions:

As I have described earlier, I start with the 068 number, then add/subtract/ignore
entries to come up with the 099 final number. I have put the +/- signs on
the original data which come from the final solution printed at the bottom.



068  Interest Income                              5.366e+10
-069 Interest and Fees on Loans                    3.951e+10
+073 Other Interest or Dividend Income             1.416e+10
+075  Total Interest Expense                       5.91e+09
+076 Interest Expense on Bank Deposits             1.4e+09
-077 Other Interest Expense                        4.51e+09
-078 Interest Expense on Debt                      4.51e+09
082  Net Interest Income                          4.775e+10
+084  Loan Loss Provision                          3.77e+09
086  Net Interest Income after Provision          4.398e+10
087 Net Interest Inc After Loan Loss Prov Growth  2.62
089 Non-Interest Income                           4.195e+10
+090 Securities Gain                               2.35e+09

091 Trading Account Income                        1.03e+09
092 Trust Income, Commissions & Fees              3.038e+10
093 Commission & Fee Income                       2.704e+10
094 Other Operating Income                        8.19e+09
095 Non-Interest Expense                          5.322e+10
096 Labor & Related Expense                       3.189e+10
097 Equipment Expense                             5.01e+09
098 Other Operating Expense                       1.463e+10
099  Operating Income                             3.272e+10
-1  3.951e+10
1   1.416e+10
1   5.91e+09
1   1.4e+09
-1  4.51e+09
-1  4.51e+09
0   4.775e+10
1   3.77e+09
0   4.398e+10
0   2.62
0   4.195e+10
1   2.35e+09
0   1.03e+09
0   3.038e+10
0   2.704e+10
0   8.19e+09
0   5.322e+10
0   3.189e+10
0   5.01e+09
0   1.463e+10
In this case Interest And Fees On Loans, add Other Interest or Dividend
Income, but you also add Total Interest Expense. There is more to
it than that, but compare this result with that below:



Here is another solution for a different bank:
068  Interest Income                              5.59e+10
069 Interest and Fees on Loans                    3.663e+10
070 Interest Income on Fed. Funds                 2.27e+09
072 Interest on Bank Deposits                     1.86e+09
-073 Other Interest or Dividend Income             1.514e+10
075  Total Interest Expense                       9.82e+09
076 Interest Expense on Bank Deposits             1.36e+09
077 Other Interest Expense                        7.37e+09
-078 Interest Expense on Debt                      6.2e+09
080 Other Borrowed Funds                          1.17e+09
082  Net Interest Income                          4.608e+10
084  Loan Loss Provision                          5.36e+09
086  Net Interest Income after Provision          4.072e+10
089 Non-Interest Income                           4.657e+10
090 Securities Gain                               4.26e+08
092 Trust Income, Commissions & Fees              4.32e+10
093 Commission & Fee Income                       4.32e+10
094 Other Operating Income                        2.95e+09
095 Non-Interest Expense                          5.272e+10
096 Labor & Related Expense                       2.998e+10
097 Equipment Expense                             3.64e+09
098 Other Operating Expense                       1.668e+10
099  Operating Income                             3.456e+10
0   3.663e+10
0   2.27e+09
0   1.86e+09
-1  1.514e+10
0   9.82e+09
0   1.36e+09
0   7.37e+09
-1  6.2e+09
0   1.17e+09
0   4.608e+10
0   5.36e+09
0   4.072e+10
0   4.657e+10
0   4.26e+08
0   4.32e+10
0   4.32e+10
0   2.95e+09
0   5.272e+10
0   2.998e+10
0   3.64e+09
0   1.668e+10

This solution subtracts Interest Expense on Debt and Other Interest
or Dividend Income from Interest Income to yield Operating Income.
The numbers work out, but the result makes no sense.

In both cases the result is exact. I let both programs run
to completion and there was only the one exact solution in
either case.

Tomorrow I will fetch a 10K report for one of these stocks
and see whether the online data correspond to the 10K.

[toc] | [prev] | [next] | [standalone]

#8270

root <NoEMail@home.org> wrote:
>
>
> The problem is with the data. The solution found is exact. The only way
> that could happen if the data are intentionally  contrived to yield an
> exact fit with the wrong entries to mask a correct fit with slightly
> different numbers. Here are two solutions:
>

I woke up last night with the realization that my program had a mistake
which prevented any perfect fit solution after the first. When I corrected
that error I found 131,000 solutions out of 10.5 billion trials in a case
that ran for 13 minutes.

[toc] | [prev] | [next] | [standalone]

#8273

On 1/5/18 12:59 AM, root wrote:
> Richard Damon <Richard@Damon-Family.org> wrote:
>> On 1/4/18 4:50 PM, root wrote:
>>> root <NoEMail@home.org> wrote:
>>>>
>>>> We'll see how the case currently running turns out. I only have to work
>>>> out the solution for four or five cases then use that solution to
>>>> unravel data for about 420 data sets.
>>>>
>>>
>>> I was surprised to find that only one solution exists to the problem.
>>> I let the program run through all cases and it finished in a little
>>> over 4 minutes. It ran through 3,486,784,401 (3.5B) cases.
>>>
>>> I really don't like the results. I'll have to ponder that before
>>> continuing.
>>>
>>
>> Due to the rounding issue, one may be a bit better than another, but you
>> should be able to replace any running sum by the values that make it up,
>> as they should be about equal, as that was part of the definition of
>> being a running sum.
>>
>> If that isn't happening, then you are just running into the issue of an
>> under constrained fitting operation, you can get an answer, but it might
>> not really mean anything.
> 
> 
> The problem is with the data. The solution found is exact. The only way
> that could happen if the data are intentionally  contrived to yield an
> exact fit with the wrong entries to mask a correct fit with slightly
> different numbers. Here are two solutions:
> 
> As I have described earlier, I start with the 068 number, then add/subtract/ignore
> entries to come up with the 099 final number. I have put the +/- signs on
> the original data which come from the final solution printed at the bottom.
> 
> 
> 
> 068  Interest Income                              5.366e+10
> -069 Interest and Fees on Loans                    3.951e+10
> +073 Other Interest or Dividend Income             1.416e+10
> +075  Total Interest Expense                       5.91e+09
> +076 Interest Expense on Bank Deposits             1.4e+09
> -077 Other Interest Expense                        4.51e+09
> -078 Interest Expense on Debt                      4.51e+09
> 082  Net Interest Income                          4.775e+10
> +084  Loan Loss Provision                          3.77e+09
> 086  Net Interest Income after Provision          4.398e+10
> 087 Net Interest Inc After Loan Loss Prov Growth  2.62
> 089 Non-Interest Income                           4.195e+10
> +090 Securities Gain                               2.35e+09
> 
> 091 Trading Account Income                        1.03e+09
> 092 Trust Income, Commissions & Fees              3.038e+10
> 093 Commission & Fee Income                       2.704e+10
> 094 Other Operating Income                        8.19e+09
> 095 Non-Interest Expense                          5.322e+10
> 096 Labor & Related Expense                       3.189e+10
> 097 Equipment Expense                             5.01e+09
> 098 Other Operating Expense                       1.463e+10
> 099  Operating Income                             3.272e+10
> -1  3.951e+10
> 1   1.416e+10
> 1   5.91e+09
> 1   1.4e+09
> -1  4.51e+09
> -1  4.51e+09
> 0   4.775e+10
> 1   3.77e+09
> 0   4.398e+10
> 0   2.62
> 0   4.195e+10
> 1   2.35e+09
> 0   1.03e+09
> 0   3.038e+10
> 0   2.704e+10
> 0   8.19e+09
> 0   5.322e+10
> 0   3.189e+10
> 0   5.01e+09
> 0   1.463e+10
> In this case Interest And Fees On Loans, add Other Interest or Dividend
> Income, but you also add Total Interest Expense. There is more to
> it than that, but compare this result with that below:
> 
> 
> 
> Here is another solution for a different bank:
> 068  Interest Income                              5.59e+10
> 069 Interest and Fees on Loans                    3.663e+10
> 070 Interest Income on Fed. Funds                 2.27e+09
> 072 Interest on Bank Deposits                     1.86e+09
> -073 Other Interest or Dividend Income             1.514e+10
> 075  Total Interest Expense                       9.82e+09
> 076 Interest Expense on Bank Deposits             1.36e+09
> 077 Other Interest Expense                        7.37e+09
> -078 Interest Expense on Debt                      6.2e+09
> 080 Other Borrowed Funds                          1.17e+09
> 082  Net Interest Income                          4.608e+10
> 084  Loan Loss Provision                          5.36e+09
> 086  Net Interest Income after Provision          4.072e+10
> 089 Non-Interest Income                           4.657e+10
> 090 Securities Gain                               4.26e+08
> 092 Trust Income, Commissions & Fees              4.32e+10
> 093 Commission & Fee Income                       4.32e+10
> 094 Other Operating Income                        2.95e+09
> 095 Non-Interest Expense                          5.272e+10
> 096 Labor & Related Expense                       2.998e+10
> 097 Equipment Expense                             3.64e+09
> 098 Other Operating Expense                       1.668e+10
> 099  Operating Income                             3.456e+10
> 0   3.663e+10
> 0   2.27e+09
> 0   1.86e+09
> -1  1.514e+10
> 0   9.82e+09
> 0   1.36e+09
> 0   7.37e+09
> -1  6.2e+09
> 0   1.17e+09
> 0   4.608e+10
> 0   5.36e+09
> 0   4.072e+10
> 0   4.657e+10
> 0   4.26e+08
> 0   4.32e+10
> 0   4.32e+10
> 0   2.95e+09
> 0   5.272e+10
> 0   2.998e+10
> 0   3.64e+09
> 0   1.668e+10
> 
> This solution subtracts Interest Expense on Debt and Other Interest
> or Dividend Income from Interest Income to yield Operating Income.
> The numbers work out, but the result makes no sense.
> 
> In both cases the result is exact. I let both programs run
> to completion and there was only the one exact solution in
> either case.
> 
> Tomorrow I will fetch a 10K report for one of these stocks
> and see whether the online data correspond to the 10K.
> 

One issue is your solution method allows for nonsense results. One way 
to think about it is that ignored items should only be used for running 
sum entries (in which case there should be a set of values that at least 
come close to summing to them), or if you are including the running sum, 
then the items used to build it should be ignored.

You may want to prune down your solution set with this sort of rule and 
see if you get something that makes more sense, as opposed to just some 
random opportunistic gotcha.


I will add that a financial report like this raises big red flags to me 
and if I got something like this from an accounting firm I would demand 
they do it right before paying, and if this was for something I was 
thinking of investing in, move away from it as fast as possible.

[toc] | [prev] | [next] | [standalone]

#8274

Richard Damon <Richard@Damon-Family.org> wrote:
>
> You may want to prune down your solution set with this sort of rule and 
> see if you get something that makes more sense, as opposed to just some 
> random opportunistic gotcha.

Exactly how I resolved the problem. I was taking too many entries at
once which, obviously now, leads to too many "solutions". By proceeding
in smaller steps I am able to identify redundant entries. I have now
resolved the breakdown of the income statements.
>
>
> I will add that a financial report like this raises big red flags to me 
> and if I got something like this from an accounting firm I would demand 
> they do it right before paying, and if this was for something I was 
> thinking of investing in, move away from it as fast as possible.

The material I showed came from an online source which provides data
for public companies. These data are available from Yahoo, Google, Nasdaq
and other sources. I sympathize with these sources because I have tried
to extract data from 10K reports and the format variation make the task
very difficult - too difficult for me.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.programming

csiph-web