DCC++ EX Software Thread

David Cutting Apr 8, 2020

  1. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    Thank you both for all your work! This was working, so we can back up and see what needs to happen. I'll post RAM usage for the Mega Sunday.
     
  2. Mani

    Mani TrainBoard Member

    76
    15
    4
    @FlightRisk Here are my findings:

    SERIAL MONITOR:
    1. Serial Monitor doesn't shows any error.
    2. The lights on the Shield responds to ON/OFF/DEBUG MODES (<1>, <0>, <D>) commands. For ON/OFF commands there no output in Serial monitor window except it displays the welcome message again. (Classic used to display<p1>).
    3. Screenshot Below (CONFIG & SERIAL MONITOR)
      2020-05-03 10_29_13-.png
    JMRI DCC++ TRAFFIC MONITOR:
    1. PanelPro - Power state is unknown. Trains not taking commands. But they are showing in traffic monitor. The light go ON/OFF when I use Power panel but indicator remains in UNKNOWN state.
    2. DecoderPro - All the above applies. Roster identification is not working when I click on IDENTIFY button. "Programmer in use" error. (I am not using it for any thing).
      NOTE: I observed only PROG track power is ON when I switch on from the DecoderPro but in PanelPro both tracks or switched On. I believe this is a feature.
    3. SCREENSHOT:
      2020-05-03 10_18_35-.png
    Let me know if this helps or any other changes/combinations you want me to try.

    Thanks,
    Mani
     
  3. haba

    haba TrainBoard Member

    78
    32
    10
    Just to clarify, the code "as is" did not break the bootloader, I did first add some debugging statements. Then it went unhappy with verify flash errors after flash writes. However I even don't know how that can be possible as I understood that the bootloader should be protected (right?). But it's a little too much of a coincidence if it would have broken for some other reason there and then.

    Here are some snipplets from my scrollback buffer while still the EX code was in the UNO:

    <s><p1 MAIN><p1 ><iDCC++ BASE STATION FOR ARDUINO UNO / ARDUINO MOTOR SHIELD: V-2.0.1 / May 3 2020 00:24:

    There is the name missing after p1

    <F><iDCC++ BASE STATION FOR ARDUINO UNO / ARDUINO MOTOR SHIELD: V-2.0.1 / May 3 2020 00:24:47>

    <F> does not answer with <f VALUE>

    Then < > (<SPACE>) did not produce a newline

    So I think everything involving CommManager::printf was broken. EDIT: <---- Hahaha, That rendered a smiley

    Another thing was that at one point one of the outputs was "half on", only one of the two LED was lit. Which is very strange as that means that the inputs of the motor shield were not at all at the values they should be. I don't know how one can get into that state.

    Well, I learned how to restore a boot manager with another UNO.

    Regards,
    Harald.
     
  4. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    Thank you for all that work. I may need your notes on restoring my bootloader when my Uno gets here. On my Mega, this what the compiler says at build time:

    21346 bytes program (31k avail w bootloader)
    1149 globals (2k SRAM avail)
    <F> command f6400

    I turned power on and off, ran the <s> command and nothing there changed free memory. It was always 6400 on the Mega. I do think this is a memory error since it works perfectly on the Mega. I'll remove strings and things. I wanted to see about a way to incorporate how you were handling the separate tracks and their different current needs. "registering" the boards was a good way to do this, but it may be too much. The code worked before, but I guess all the testers just had megas! I thought the free memory would have changed after entering a few commands. Maybe I have to issue a throttle command so something puts things in the registers. Just wanted to give a quick reply. I'll test more after breakfast ;)
     
  5. ardsuppi

    ardsuppi TrainBoard Member

    33
    8
    3
    I got this at one point too. Even alternate flashing of MAIN leds.
     
  6. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    I don't have an Uno to test yet, one on the way thanks to a generous board member, but compiling as I I had one, I see this:

    Sketch uses 20700 bytes (64%) of program storage space. Maximum is 32256 bytes.
    Global variables use 1129 bytes (55%) of dynamic memory, leaving 919 bytes for local variables. Maximum is 2048 bytes.

    Time to count up all the local variables and strings.
     
  7. haba

    haba TrainBoard Member

    78
    32
    10
    You wrote: <F> command f6400

    Ok, that's on a Mega which has 6*1024=6144 bytes more RAM than the UNO. So if we do 6400-6144 then the same number should be 256 bytes free at the same place when on the UNO. That value of free space is when you do the calculation in SerialCommand.cpp before you call the CommManager::printf() to actually write out that value. When you do that you first put return adress on the stack to be able to return from the funcion and then you allocate local vars of that function. That starts with "char buf[256] = {0};" and here the remaining RAM was filled and we get a heap/stack collission. Then there are more function calls from that point that build on the stack a little more. With a bit of luck that results in a reset. When umlucky you get corrupted stuff all over the heap.

    Regards,
    Harald.
     
    KC Smith likes this.
  8. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    Yes. I noticed that ;) At some point we are going to have to decide when the Uno can't be supported for new features. We can get all we've done so far other than perhaps the com manager and motor board manager. They aren't critical either, but again, we need to determine when the platform is holding us back.

    What was strange though, and I will test again, is that this number did not change on my Mega. As soon as I hit <s> and to load the status strings and entered a throttle command which uses a register, it should have gone down.
     
    Last edited: May 4, 2020
  9. haba

    haba TrainBoard Member

    78
    32
    10
    But one does not need to allocate a 256 bytes buffer to print status messages. One does not need always to reserve space for 10 (if I saw that right) motor boards. This code has been written like you do for a big computer with OS and more RAM. With the Arduino IDE it's easy to forget your target. When you program in assembler (no, I don't want to) that's not that easy to miss ;-)

    The biggest RAM thieves in the original code are strings and the register list. Then you need to make different features that need RAM optional to compile in, so that the user can decide.

    The registers are statically heap allocated in RegisterList::RegisterList(int maxNumRegs). Until you use them they are just empty. To allocate them on demand would be a pain and no real benefit because either you have RAM for X registers (and then you can allocate that space from the beginning) or you don't.

    Regards,
    Harald.
     
    KC Smith likes this.
  10. KC Smith

    KC Smith TrainBoard Member

    109
    111
    12
    Harald and FlightRisk,

    I down loaded EX last week on an Uno and it failed to turn on and got the dancing Led's.
    Might need to reload boot loader before I try testing again after your next release.

    ''this thought of temporarily removing Comminterface.ESP to save memory space edited out'

    Also, in DCCppEX can we add these lines so we have a better idea of what we're working with when running code on serial monitors?

    void showConfiguration() {
    Serial.print("\n*** DCC++EX CONFIGURATION ***\n");
    Serial.print("\n* Released 4/7/20 *\n");

    This might improve the testing commentary feed back If we're all on the same page & testing the same code release over time.

    Regards,
    Kevin
     
    Last edited: May 4, 2020
  11. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    On the development version, I did exactly that, added EX to the string so we know what we are working with. The version starts with 2.x.x also which is a tell. I also think that these strings are unnecessarily long. The first thing you should see if you don't comment it out when you ope a serial monitor is displayed from this code:

    "<iDCC++ EX BASE STATION FOR ARDUINO %s / %s: V-%s / %s %s>", ARDUINO_TYPE, MOTOR_SHIELD_NAME, VERSION, __DATE__, __TIME__);

    QUESTIONS:

    What other than JMRI and Rocrail have interfaces that send and receive responses from DCC++ EX?
    Do any of them that have code that looks in a status string for a particular phrase?
    This function, <s> and <F> all return values with no space between the descriptor and the value. This is inconsistent. Again not sure I will break something in JMRI or something else by doing this:

    <i DCC...> and <f 1848> instead of <iDCC++...> and <f1848>

    To save a few bytes, what happens if we change DCC++ EX to DCCEX or DCC++EX? And/or remove "FOR ARDUINO". Is that really necessary? If you upload this code to an Arduino, you certainly know it only runs on an Arduino. It doesn't seem to offer anything, yet costs us 11 bytes. For that matter, do we need "BASE STATION"? Isn't DCC++ EX the product? I think it leaves open the possibilty there can be a DCC++ EX Controller or a DCC++ EX Booster or whatever, but not sure that will happen or not adding "base station" to the description would affect any of that. It is not as easy as putting F( in front of the string since this and some other messages are broadcast through commanager to go to all the listeners. It would be nice to have JMRI connected serially and a handheld controller running wifi directly to the base station and not through JMRI running on another computer.

    I already got 300+ bytes back in SRAM just by playing with strings that could have F( added to them. Atani created the comm manager and allocated 256 bytes. I was looking at that myself to see everything that was printed in it to see what the maximum string was. So far, memory stays the same until I used a throttle command, but only for the first loco. Once the registers are created, it seems to remain static. The benefit of using functions as we have is that the variables only take up space during execution of the function. Once the return value is pulled off the stack for the program to continue out of the function, the space is freed.
     
    KC Smith and Sumner like this.
  12. haba

    haba TrainBoard Member

    78
    32
    10
  13. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    I will have an Uno in about 3 days. If anyone wants to test the development branch, let me know. I freed up 300 to 400 bytes just by changing a few integers to Uint8_t (sense pins) and using the the F( PROGMEM macro to put more strings into the program memory. I also shortened a few text strings. The last thing I did was reduce the 256 byte buffer for the CommManager to 128. So cut by half. Though that memory is only used when it sends message responses and then freed again. I think the the buffer can be made even smaller. Here is the longest message:

    <i DCC++ EX BASE STATION MEGA / ARDUINO MOTOR SHIELD: V-2.1.1 / May 4 2020 15:54:21>

    Notice I removed "FOR ARDUINO". It will just say BASE STATION UNO, etc. Do I need to put it back? I noticed something else, but the compiler may pick it up anyway, there is repeated text that I wonder if putting it into an #define would save space. "DCC++ EX BASE STATION" is repeated in 3 different print statements. Also "\n" is repeated all over the place as in: CommManager: Printf("\n"). So I'll try replacing this unless someone knows for sure it won't help.
     
    Last edited: May 4, 2020
    Mani and KC Smith like this.
  14. haba

    haba TrainBoard Member

    78
    32
    10
    #defines are expanded before compile and do not safe space in itself because it's expanded to several strings again.

    Compare the following two ways to allocate strings:

    #define MESSAGE "Arduino"
    printf( MESSAGE" starts");
    printf(MESSAGE" stops" );

    and

    #define MESSAGE "Arduino"
    printf(MESSAGE);
    printf(" starts");
    printf( MESSAGE);
    printf(" stops" );

    The first one will allocate two long strings and the second one three short ones because the #define is evaluated before the compiler gets into action and the compiler will not see common substrngs. The compiler will however dectect two 100% identical strings like prinrf("xxx") at one place and char *c = "xxx" at another and allocate those only once.

    You can test interactively at
    https://godbolt.org/z/ZKFVbh
    This is only the bare comiler without the whole IDE but in principle the same.

    Regards,
    Harald.
     
    KC Smith and FlightRisk like this.
  15. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    What a great tool! I never heard of compiler explorer before. It spits out the assembly and shows exactly what is happening. Going to play now, see you in a week ;)
     
  16. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    @ardsuppi It is working on the Uno now. You will notice a lot of changes to the code. A lot of motherboards for one thing and a different way to deal with current in actual milliamps (though for now we still report it to jmri as a pin reading). I need to do read through all the changes one more time, and test in JMRI and RocRail. As you can tell, my Uno came today ;) If you like, you can go here and test the development version:

    https://github.com/DCC-EX/BaseStation-EX/tree/development

    Make sure you see "development" in the left side "branch" button and then get the link on rights to get the zip like you did before. This zip should have "development" in the name. The short circuit protection on an Arduino Motor Board should trip at 1500mA or 1.5A. That is configurable. Track current is now the NMRA standard 250mA. That is also configurable since some loco decoders may not be "standard" and could pull a little more than that ;) The more people and equipment we test with, the better.
     
    KC Smith likes this.
  17. KC Smith

    KC Smith TrainBoard Member

    109
    111
    12

    Thanks FlightRisk,

    DCCppEX_development compiled and uploaded just fine on a original Arduino Uno/Motor shield, IDE 1.8.12
    Quick test on JMRI 4.18+R37ad3d0 with EngineDriver (android) throttle also works fine.. So Far.

    compile note: used 20776 bytes (64% of 32256 max bytes
    and 773 (34%) of 2048 dynamic memory

    Good Job!

    Are there specific areas you want us to test out for you?

    Regards,
    Kevin
     
  18. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    Just general operation, perhaps reading CVs on the programming track. I am going to test the current sensing to see what the arduino says, vs. what a meter says. But to do that, you have to change code since I didn't want to break JMRI. The code in response to the <c> command that reports <a xxx> still gives a pin reading, but the code is there to spit out <a pin milliamps trigger max>. It is commented out in 2 places and the other line of code left in. I should have put a temp code in there to spit it out that way for testing. I may do that because ultimately, measuring current on different layouts and different motorboards will give us a more accurate "sense" or any adjustments we need. Bad pun ;

    If the overcurrent protection is tripped, the output in the serial monitor is <p2 track> where track is PROG or MAIN. Normally you would see <p0> or <p0 track>. If a track shuts off, it is supposed to come back on again automatically.
     
    KC Smith likes this.
  19. KC Smith

    KC Smith TrainBoard Member

    109
    111
    12
    Thanks for putting in the additional work.
    It would be nice to have a delayed auto restart after a triggered short.

    Regarding current trigger value in Feb 2016 when I installed the first release it was set to 300 for his N scale 12 volt system and it would trip early with four or more HO scale sound Engines. At that time Gregg B suggested I raise it to 600 which I did and it worked fine 8 or more HO engines before tripping. I estimate that was around 1780 millamps.
    I would be interested to see if other HO scale users find that they have early trips when tripMillamp set to 1500 in the base config.
    I plan on raising the base config up a little maybe to 1650 as we go along.

    Rgards,
    Kevin
     
  20. FlightRisk

    FlightRisk TrainBoard Member

    548
    237
    14
    Funny, I had a couple of sentences about this and just cut them in the last message :) I was going to ask what people thought. I'll probably put it as a task in GitHub and ask in the other thread what people think. Now the check runs every time through the loop and because the constant that controls the time is set for 1ms, it polls pretty fast ;) Separating the main current check and a larger delay to turn it back on could easily be done.

    The default value for the Arduino Motor Shield in DCC++ Classic is/was 300. That corresponds to about 890mA. EX's default is 1500mA. That should be enough. Without cooling, that board really can't put out 2A continuously. YMMV :) For that board, our conversion factor is 2.96. So multiply pin reading by that to get current in milliAmps. I have a table where someone did one with actual values that was a little different than these theoretical ones.

    There is also the constant CURRENT_SAMPLE_SMOOTHING that affects the reading. It is different for the different processors. Haba is a SBC purist and I appreciate that. I like some of his ideas like not using floating point math so that we save memory with integer math. He also has been experimenting with different methods of handling the current sense issues. And the new code supports other current sense devices. The MAX471 because it is designed just for current sense is great because it outputs 1V per Amp, the way nature intended ;) We also have to consider how the device outputs to the pin. We might not have good resolution on a high current device trying to read only 0-1023 steps on a pin and using that to read low currents. Lots to work on.
     

Share This Page