great progress

Back again to give you some further information on this project. It took a while to test a few proof of concepts. First I adapted the i2c input and output routines to get it stroked with the specification of the PCA9698. I tried both ways ; doing a simple write() and doing ioctl() using the i2c message structure. Both api calls seems to work properly. I wrote a small test program which includes the input as well as the output functionality. This program ran for 24h without any odd behaviour. So now I'm pretty sure that the IO board works as it was meant to be by design. Yesterday I worked a hole day to get the AC97 codec from Cirrus up and running. A few technical conversations can be seen at following link:

http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=62973

I'd play quit a lot with kernel builds and tests via the procfs systeem. Finally at 21h I managed to play my first sound with aplay as the driving program. It was also indispensable to have the UI alsamixer installed on the NGW. This facilitates the setup of all the codec channels. It's possible todo it via the procfs but quit complex because you must make the link to registers in the specification all the time and of course easy to make a small mistake in setting the bitmasks accordingly. For the moment I only tested the wav format. I will do some quick tests in MP3 using madplay too.

What else do I have the test to be able to say "now I start programming on my new domotic system" ? ( this will be a title some day of one of my blogs here ;-)

1. Writing a test for the DAC (also i2c driven)
2. Add interrupt masking to the pca input
3. Writing a test to catch the interrupt in user space
4. Evaluate the level based interrupt and filter (re entrance)
5. Writing a "covered all" test and let it run +24h
6. Extend this program including a exec call the sound program
7. Making a documentation
8. Compiling / installing / testing with lighthttpd

If these above steps are all behind me than I can proceed with the domotic logic for the final program. I'm still not sure if I will integrate this into web based design. This will depend on the results of step 8.

The horizon is something you can never reach nevertheless continue to try !

connected to NGW and up and running

At last everything on the i2c side works as it should be. Special thanks to Manes for his effort in replacing the components. In fact we had only one error on the board which was a wrong position of the jumper for the external extender pull up, introduced from the start. The rest of our problems was pure invoked by causality reasons. I started a bit of a panic and did some wrong measurements on the board that resulted in fucking up almost all the i2c chips. It was as simple as that. I tested quickly the inputs and outputs already and seems to be ok. In depth testing will be for this coming week. I will post a full test report here at my blog later.

We are back into business finally !

removing the smd pca9698

So here we continue our struggle to solve the hardware problems on the IO board. I took the board with me to my work because of the lack of the expensive equipment at home. On the sides I show you some pictures in sequence on how and with what the removal took place:

Arrival in the lab

Removal pca with hot air

Capillary cleaning

Removal device for SMD and BGA

Cleaning device

The Weller lab

Next episode will be soldering the new PCA9698 and extender on the IO board and testing with the low level assembler routines for the I2C proof of concept.

the hell of PCA9698

Here we go with the continuation of our doom story. In the previous blog I told you that Manes would test the i2c with his logic analyzer. And he kept his promise here are the results.
First of all he'd checked the SCL/SDA signal level/transitions with respect to the i2c general and PCA specific specification. It seemed to be ok except that some ACK didn't arrive, as it had to be in the read process. This was odd and suspicious. In the process of testing with the only good PCA he made a mistake in putting 12V on the inputs. So guess what another chip fucked up !

But more important he tested his bit-banging algorithm on a simple PCF and all signals appeared like in specifications. Conclusion is clear the PCA output chip was also fucked up earlier together with the input PCA. The ordering of 2 new PCAs and one extender is now in progress. We expect to test again with the new chips till the end of this week. This time we take things gradually no doubt about that.

In fact the reason why all chips were fucked up isn't yet totally clear in my opinion. When I first connected the NGW to the IO board, the chips were just soldered on the board and didn't see any of supply. At that time we had already the voltage drop. (1V) So the question here is what caused this drop ? It's true after the uncertainty of all that I started to play with board and did some measurements (eg. capacity); which in fact were not that smart. So it could be that in the process of all that some of the chips suffered badly of unwanted supplies. Nevertheless the previous question remains. I definitely want an answer on that. Maybe things become clearer when we replace the components gradually. I propose replacing in following order and test via bit-banging assembler program:

PCA output
PCA input
Extender

Suppose that after replacing all components everything is working back again than possibly the reason of malfunction was that one chip was already broken before the initial testing with the NGW. But these are speculations for the moment so I leave this conclusion till the end of the week. If tests with 8bit debug AVR went well, I can test with NGW linux bit banging algo's. If these work I finally can build the domotics. If these don't work I probably have to make changes on the existing bit banging algo or considering writing my own device driver from scratch.

In order to remove the two smd PCAs from the board a special suction tool is needed otherwise the board will be irrevocable damaged. This tool, I can use from the firm I work for luckily. I will take some shots of before, while and after for the blog because perhaps it will be a point of change in our luck so far.

error trackings

In the previous blog I mentioned the i2c problems when connecting the IO board to the NGW. The TWI lines of the AVR32 stayed at 3.3V so I had to find a workaround for that in the first place. I rebuild the kernel with the GPIO i2c instead of using the atmel twi driver. First it didn't work when simply selecting the GPIO and deselecting the twi in the menuconfig. I had to comment out the GPIO defines in the setup.c file so it was always taken. This must be a small config bug somewhere in the buildroot I use. After that the GPIO was proper recognized and i2c-0 appeared in the dev list. Even more, the device name was the same as the twi driver so any modification on my user space programs were not necessary. This is a good example of portability if you ask me. I checked out the waves on the bus with my scoop and this seems to be as it should be. Happy for sure because I could continue using the NGW to connect to the IO board.

In the meanwhile Manes started to dive into the IO board measurements as you can see on the picture in this blog. First of all he discovered the same voltage drop when connecting his debug AVR (8 bit) master. He disconnected the extender , put on the board for future use , and start running his output test program for the PCA. The 40 leds started to blink at last ! But he also saw that the signal level on SDA was only ca 2V. So his test program for the DA converter failed. He decreased the pull up a bit and the DA converter test was also running fine. I visited Manes one evening and connected the NGW , now with GPIO, to board in order to check if I could also play with the 40 leds on the output. But none of my tests seemed to work. So making use of the linux i2c driver was indeed not the same as the assembler (also bit-banging) test program of Manes. further investigations needed here.

Just one more test to go : the inputs. First try outs of reading a simple pin change and put it onto the output leds failed. Second test with printing the input read outs constantly on the serial console failed with same behaviors. Manes took the initiative to cut the SDA line of the PCA input. And guess what : our signal drop was gone ! Both SCL/SDA had a proper 3.3V. Than the idea that the PCA input was fucked up was more or less clear. After that, he started to run his input tests on the output PCA. Until now he didn't succeed in getting this working. Even with the addition of the repeated start condition. He will investigate this with logic analyzer this afternoon.

The fucked up list until now:

• AVR32 twi lines
• Extender P82B96
• 1 PCA9698

We have to wait now for the results of the logic analyzer. The moment we have a reference level/transition i2c construction for the PCA to work in both read an write will be very important to make decisions for the future of this project. If this indeed is a kind of a special signal, than changes to the default GPIO driver from linux will be indispensable.

Stay tuned for the thrilling moments to come...

NGW meets IO board

The first thing I did was checking the position of the control signals on J5. I changed the configfs mask and output accordingly. I also added two lines to the script to accomplish the reset for the audio codec and pca. And than the moment arrived to switch on the power of both boards and let the linux kernel boot up. Ok so far so good power led lights up and off we go...

I noticed already in the boot sequence that the audio codec was still not recognized as an ALSA device. This could be due the fact the reset executes after the loading of the kernel module. So I tried with a warm NGW reboot after reset still the same. This means that testing the sound was not possible because I didn't have a DSP in the device list present. Still not sure if the message printed by the audio driver was applicable on the AC97 I posted a question on the freaks forum.

I let the AC97 behind I started to the focus on the I2C communication. I adapted the C program with the proper address of the output PCA. I first tried with a simple write of 6 bytes 1 control and 5 outputs hoping the acks where properly formed. This was not a success. The second thing I tried was sending them using the I2C message struct, which normally takes care of the acks in between before sending a stop condition. Still no success. Last thing I could do was taking back the I2C bus scan routine and see if the board controller was still connected. This test was terrible all 128 address tests failed in giving me the "connection time out" as an error message. If I cut all the flat cables to the IO board the test performed as it should be. At this point 4.7K resistors were used as pull up defaults from NGW. I tried in putting 3.3K in parallel on the IO board because the specification of the PCA recommends even 1.6K pull up. Still no success. After some initial checks on the PCB layout Manes discovered that the RESETs of the PCAs were not connected to the flat cable. So the second wire patch was a fact. Again restart with all tests still the same. Sounds like it had something todo with board itself and it was a good moment to start thinking on scoop measurements. The signal on both SCL and SDA were terribly bad, when the IO board was connected to NGW. I'm talking here of signal droppings from 3.3V to 1V. Of course this was the reason of the malfunction here. The I2C bus was to heavily loaded for the NGW open drain output.

The twi output of the NGW stops after a few seconds when testing a write in loop. The output remains static on SCL ca 0V and SDA ca 1V. And now it's even doing nothing anymore its just stays on 3.3V. It's obvious that these two outputs are broken. It's a pity that on evaluations like NGW apparently nothing is foreseen to protect or buffer the IOs. It was also unexpectedly because the proof of concept with a single PCF worked well. But if the amount of connected twi devices rises the charge on the bus will do the same and modern micro controllers like AVR32 and ARM9 are not that well protected against these loads.

Probably introducing a I2C extender or signal converter (3.3 -> 5V) solves issues like that. But that I can't prove at the moment I’m afraid. As my SDA/SCL is fucked up I must find a workaround. If I can't find it the NGW board will be of no use anymore and can be trashed. The linux is still booting so I think it's only these two outputs that are bad. Buying a new one is of course the only solution if I want to continue using the atmel dedicated twi. If I can go by the GPIO driver (no atmel twi but bit-banging) and switch to some other gpio pin numbers on the connector than there' s a bit of hope left. But the reason of malfunction must still be proved. So a test with extender/converter will be indispensable. A converter is preferred over an extender when a IO board revision will be made because lack of space on the NGW to mount extender.

I'm very disappointed. Maybe Manes could avoid things like that in considering worst-case situations. On the other side it still remains a low cost development kit of less than 100€. Now more and more the question gains presence in my mind:

"Is this a good solution to serve as a 24/7 application in situations where an influence like electrical disturbance is present?"

I still have some hope though.