DECIMAL POINT HEADACHE

[siliconchip]

Hi martin

Thanks im going to have a good play around in the next couple of days i will keep you posted on what i find and where i go

Cheers bob

[siliconchip]

hi martin
right been playing and done some tests which are the decimal 3 version
first off using version decimal 3 changed the ref resistor to 1000000.0 Meg as per fluke reading, using pull up res used = 1 i tested another 1Meg res which was 1005000 Meg according to a fluke, result was 1.009 Meg so by my calcs is 4Kohms over

exactly the same test but with pull up res used now at 0 (pull up polarity changed) and reading this time 1.003 Meg which is 2 K ohms under the fluke reading

how ever when changing the ref resistor back to 267.3 for a 147.4 resistor as per fluke value i was getting 147.1 and 147.6 respectively i then tried the original decimal 1 version with both ref resistors above and found the MEG function doesnt work as we know but the low value was identical to the fluke, therefore can i presume that due to internal impedance on the anologue input it does not like seeing a value like 1 Meg hence decimal 1 version not working but your reworked decimal 3 good but decimal 1 being very slightly more accurate than decimal 3 in very low readings because of the maths needed in decimal 3 to make the Meg function work ??

cheers bob

[medelec35]

Hi Bob.
With the earlier versions, the type of the variable used for PullupResistor is UInit.
That means the maximum value is 65535, so you can not have any resistance value greater than 65k.
For higher values, you can either use a ULong or Float.
It also means the fractional part of the number is ignored.
That will also apply to Bytes, Int, UInt, long and Ulong.
Therefore 267.01 will be treated as exactly the same as 267.99 = 267
This could be why there are differences between the first and the third visions as the latest version uses a Float.
Floats will use the fraction part which will change the result if using 267.01 or 267.99.
It's really odd that the third version is less accurate.
To maintain the accuracy with the second version try two things.
Change the PullupResistor from Uint to ULong.
Finally, remove the fractional part, so you only have 267
Then for 1.52Meg use 1520000 etc.

[siliconchip]

Hi martin
As always thanks for the reply you have made good valid points and ive made rookie mistakes lol however i can now move forward abit more based on your explanations and i will keep you informed

Cheers bob

[siliconchip]

hi martin

i carried out your last recommendations , i changed the ref resistor to ULong and now this has perked version 2 up with a 1 Meg resistor as ref i can get down to 100k ohm with accurate results compared to a fluke but no good for low value ie 180 ohm but this accuracy comes back with the original 267 ref resistor, and all variations used the please connect function works correctly , plus the M, K and ohm sign show when required its now a matter of getting the right value ref resistors and tailoring to my needs, i totally overlooked when making the ref variable what to make it as you rightly pointed out changing from UInt, ive got a fair way still to go but i will keep you posted and as always a massive thank you for the help its much appreciated

cheers bob

[medelec35]

Hi Bob,
Sounds like you are making good progress.

perked version 2 up with a 1 Meg resistor as ref i can get down to 100k ohm with accurate results compared to a fluke but no good for low value ie 180 ohm but this accuracy comes back with the original 267 ref resistor

What about my suggestion of

f you want to test resistors in the meg range, then the reference resistor needs to be in the meg range and the .ReferenceResistor has to be set for the same value.
I would suggest you need three ranges of ref resistances.
Mid 100's ohm
Mid 100k's ohm
and perhaps mid megohm?
Use the ADC value to auto-select the required range via either relays, reed switches or analogue switches
starting off with the lowest ref value.
Loop: If ADC >900 then switch in a higher value resistor: Exit loop

Using reed relays to connect the desired reference resistance as analogue switches resistance could be a bit too high for want you want unless you take it into account when entering the values of the reference resistors.

[siliconchip]

hi martin

ive already dabbled with reed relays putting an analogue value through them the relays im using have a max of 150 milli ohm contact resistance and made no visible change to the values measured, i had originally used 5v non latching relays but these were terrible and the readings were quite abit above what they should be im using a ULN 2003 darlington array chip to drive them so it keeps component count down ie several transistors diodes resistors etc, ive got more experimenting to do and auto ranging is a possibility but still away to go but the basics are now coming together with your help

regards bob