davidalo

davidalo

Member Since 5 years ago

SIDISEL ELECTRONICA S.L., Salamanca

Experience Points
3
follower
Lessons Completed
5
follow
Lessons Completed
9
stars
Best Reply Awards
2
repos

17 contributions in the last year

Pinned
⚡ HTTP/Socks4/Socks5/Shadowsocks/ShadowsocksR/SSH/Redirect/Pf TCP/UDP asynchronous tunnel proxy implemented in Python 3 asyncio.
⚡ Lightweight JSON library written in C.
Activity
Nov
24
1 month ago
Activity icon
issue

davidalo issue comment ARMmbed/mbed-os

davidalo
davidalo

Mbed OS network tests may not be suitable for some cellular communications

Description of defect

I have been developing a new driver for a QUECTEL modem. I am trying to run the Mbed OS tests using the new porting, all of them works as expected except two:

  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST
  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST_NONBLOCK

I have been reviewing those tests and they perform the following actions:

  1. They send multiple UDP packets of different data and lengths to an echo server.
  2. UDP packets are received using a buffer of the maximum packet length.
  3. UDP packets are checked to be received (even in a different order)

The problem is that, this tests are assuming that the UDP socket API will pop a UDP packet (and no more). For example, if we receive three packets of 100, 300 and 200 bytes and we try to read 600 bytes, we will have to perform three read operations and read 100, 300 and 200 bytes respectively.

The problem is that, in some modems, such as in QUECTEL ones, UDP datagrams are stored as if they were a TCP stream, it does not have delimiters for the different received packets so when you try to read 600 bytes, it will return you the three packets (in the order they have been received).

I actually don't know how to approach this in order to make the component pass the tests.

Target(s) affected by this defect ?

Cellular components.

Toolchain(s) (name and version) displaying this defect ?

GCC_ARM

What version of Mbed-os are you using (tag or sha) ?

mbed-os-6.14.0

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

mbed-cli version 1.10.5

How is this defect reproduced ?

Running Mbed OS connectivity tests:

mbed test -n mbed-os-connectivity-netsocket*
davidalo
davidalo

Added some extra details.

Nov
19
2 months ago
Activity icon
issue

davidalo issue comment pyinvoke/invoke

davidalo
davidalo

Update vendored pyyaml version from 3.11 to 5.4.1

This addresses a bunch of security fixes that PyYAML has addresses since the last vendored version of the package.

Fixes #708

Follow up to #796

davidalo
davidalo

@bitprophet I believe it's the combination of having an invoke.yaml file in the project root and some rearragnments made in the standard library in Python 3.10. The actual error is AttributeError: module 'collections' has no attribute 'Hashable'.

Without the invoke.yaml file, invoke works. I guess this was just the first project I tried to use with Python 3.10.

To reproduct my setup, I'm working with the code at https://github.com/MinchinWeb/minchin.releaser and Python 3.10 on Windows 10 (although I don't think anything in Windows specific). Clone the repo, create and activate a virtual environment, and install the project (pip install .) (which will install invoke), and then run any invoke command, including invoke --list. Here is the traceback:

(.venv-310) C:\Code\minchin.releaser [master ≡]
> invoke --list
Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Code\minchin.releaser\.venv-310\Scripts\invoke.exe\__main__.py", line 7, in <module>
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\program.py", line 373, in run
    self.parse_collection()
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\program.py", line 465, in parse_collection
    self.load_collection()
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\program.py", line 702, in load_collection
    self.config.load_project()
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\config.py", line 750, in load_project
    self._load_file(prefix="project", merge=merge)
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\config.py", line 889, in _load_file
    self._set(data, loader(filepath))
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\config.py", line 909, in _load_yaml
    return yaml.load(fd)
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\__init__.py", line 72, in load
    return loader.get_single_data()
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\constructor.py", line 37, in get_single_data
    return self.construct_document(node)
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\constructor.py", line 46, in construct_document
    for dummy in generator:
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "C:\Code\minchin.releaser\.venv-310\lib\site-packages\invoke\vendor\yaml3\constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
AttributeError: module 'collections' has no attribute 'Hashable'

I am having the same issue, any updates on this?

Activity icon
issue

davidalo issue comment pyinvoke/invoke

davidalo
davidalo

invoke fail in Python 3.10 while parsing yaml configuration.

Running inv(oke) in Python 3.10 raises exceptions, while parsing yaml configuration.

How to reproduce

  1. Install invoke in Python 3.10 env.

    python3.10 -m venv .venv
    source .venv/bin/activate
    pip install invoke==1.6.0
    
  2. Make a dummy tasks.py:

    touch tasks.py
    
  3. Make a invoke.yml:

    echo 'debug: true' > invoke.yml
    
  4. Run invoke

    inv
    

Expected behavior

Run without error

Actual behavior

fail with exception

(.venv) ➜  invoke-test inv
Traceback (most recent call last):
  File "/tmp/invoke-test/.venv/bin/inv", line 8, in <module>
    sys.exit(program.run())
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/program.py", line 373, in run
    self.parse_collection()
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/program.py", line 465, in parse_collection
    self.load_collection()
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/program.py", line 702, in load_collection
    self.config.load_project()
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/config.py", line 750, in load_project
    self._load_file(prefix="project", merge=merge)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/config.py", line 889, in _load_file
    self._set(data, loader(filepath))
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/config.py", line 912, in _load_yml
    return self._load_yaml(path)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/config.py", line 909, in _load_yaml
    return yaml.load(fd)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/__init__.py", line 72, in load
    return loader.get_single_data()
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/constructor.py", line 37, in get_single_data
    return self.construct_document(node)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/constructor.py", line 46, in construct_document
    for dummy in generator:
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/tmp/invoke-test/.venv/lib/python3.10/site-packages/invoke/vendor/yaml3/constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
AttributeError: module 'collections' has no attribute 'Hashable'
davidalo
davidalo
Activity icon
issue

davidalo issue comment pyinvoke/invoke

davidalo
davidalo

Invoke is broken for Python 3.10

As I saw in #820, there are more people who are having the same issues with Invoke and Python 3.10. As far as I could see, invoke seems to be broken in Python 3.10. It seems that Python has moved some classes from collections to collections.abc. This produces ImportError exceptions y some invoke features.

To reproduce this issue, you can see #820. In addition to this, I have tried it using python:3-alpine3.13 docker and adding a simple task. The result is similar to #820:

/ # inv -l
Traceback (most recent call last):
  File "/usr/local/bin/inv", line 5, in <module>
    from invoke.main import program
  File "/usr/local/lib/python3.10/site-packages/invoke/__init__.py", line 2, in <module>
    from .collection import Collection  # noqa
  File "/usr/local/lib/python3.10/site-packages/invoke/collection.py", line 4, in <module>
    from .util import six, Lexicon, helpline
  File "/usr/local/lib/python3.10/site-packages/invoke/util.py", line 1, in <module>
    from collections import namedtuple, Iterable
ImportError: cannot import name 'Iterable' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)

Invoke version:

/ # pip show invoke
Name: invoke
Version: 1.6.0
Summary: Pythonic task execution
Home-page: http://docs.pyinvoke.org
Author: Jeff Forcier
Author-email: [email protected]
License: BSD
Location: /usr/local/lib/python3.10/site-packages
Requires: 

I think this can be fixed by checking the Python version before importing elements from collections or using a try/except statement. I can submit a PR if you want.

davidalo
davidalo

This is fixed in master version, but we would need a new release to fix them in production.

Activity icon
fork

davidalo forked pyinvoke/invoke

⚡ Pythonic task management & command execution.
davidalo BSD 2-Clause "Simplified" License Updated
fork time in 2 months ago
Activity icon
issue

davidalo issue pyinvoke/invoke

davidalo
davidalo

Invoke is broken for Python 3.10

As I saw in #820, there are more people who are having the same issues with Invoke and Python 3.10. As far as I could see, invoke seems to be broken in Python 3.10. It seems that Python has moved some classes from collections to collections.abc. This produces ImportError exceptions y some invoke features.

To reproduce this issue, you can see #820. In addition to this, I have tried it using python:3-alpine3.13 docker and adding a simple task. The result is similar to #820:

/ # inv -l
Traceback (most recent call last):
  File "/usr/local/bin/inv", line 5, in <module>
    from invoke.main import program
  File "/usr/local/lib/python3.10/site-packages/invoke/__init__.py", line 2, in <module>
    from .collection import Collection  # noqa
  File "/usr/local/lib/python3.10/site-packages/invoke/collection.py", line 4, in <module>
    from .util import six, Lexicon, helpline
  File "/usr/local/lib/python3.10/site-packages/invoke/util.py", line 1, in <module>
    from collections import namedtuple, Iterable
ImportError: cannot import name 'Iterable' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)

I think this can be fixed by checking the Python version before importing elements from collections. I can submit a PR if you want.

Nov
17
2 months ago
Activity icon
issue

davidalo issue comment ARMmbed/mbed-os

davidalo
davidalo

Mbed OS network tests may not be suitable for some cellular communications

Description of defect

I have been developing a new driver for a QUECTEL modem. I am trying to run the Mbed OS tests using the new porting, all of them works as expected except two:

  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST
  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST_NONBLOCK

I have been reviewing those tests and they perform the following actions:

  1. They send multiple UDP packets of different data and lengths to an echo server.
  2. UDP packets are received using a buffer of the maximum packet length.
  3. UDP packets are checked to be received (even in a different order)

The problem is that, this tests are assuming that the UDP socket API will pop a UDP packet (and no more). For example, if we receive three packets of 100, 300 and 200 bytes and we try to read 600 bytes, we will have to perform three read operations and read 100, 300 and 200 bytes respectively.

The problem is that, in some modems, such as in QUECTEL ones, UDP datagrams are stored as if they were a TCP stream, it does not have delimiters for the different received packets so when you try to read 600 bytes, it will return you the three packets (in the order they have been received).

I actually don't know how to approach this in order to make the component pass the tests.

Target(s) affected by this defect ?

Cellular components.

Toolchain(s) (name and version) displaying this defect ?

GCC_ARM

What version of Mbed-os are you using (tag or sha) ?

mbed-os-6.14.0

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

mbed-cli version 1.10.5

How is this defect reproduced ?

Running Mbed OS connectivity tests:

mbed test -n mbed-os-connectivity-netsocket*
Activity icon
issue

davidalo issue ARMmbed/mbed-os

davidalo
davidalo

Mbed OS network tests may not be suitable for some cellular communications

Description of defect

I have been developing a new driver for a QUECTEL modem. I am trying to run the Mbed OS tests using the new porting, all of them works as expected except two:

  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST
  • mbed-os-connectivity-netsocket-tests-tests-netsocket-udp : UDPSOCKET_ECHOTEST_BURST_NONBLOCK

I have been reviewing those tests and they perform the following actions:

  1. They send multiple UDP packets of different data and lengths to an echo server.
  2. UDP packets are received using a buffer of the maximum packet length.
  3. UDP packets are checked to be received (even in a different order)

The problem is that, this tests are assuming that the UDP socket API will pop a UDP packet (and no more). For example, if we receive three packets of 100, 300 and 200 bytes and we try to read 600 bytes, we will have to perform three read operations and read 100, 300 and 200 bytes respectively.

The problem is that, in some modems, such as in QUECTEL ones, UDP datagrams are stored as if they were a TCP stream, it does not have delimiters for the different received packets so when you try to read 600 bytes, it will return you the three packets (in the order they have been received).

I actually don't know how to approach this in order to make the component pass the tests.

Target(s) affected by this defect ?

Cellular components.

Toolchain(s) (name and version) displaying this defect ?

GCC_ARM

What version of Mbed-os are you using (tag or sha) ?

mbed-os-6.14.0

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

How is this defect reproduced ?

Running Mbed OS connectivity tests:

mbed test -n mbed-os-connectivity-netsocket*
Nov
16
2 months ago
push

davidalo push davidalo/parson

davidalo
davidalo

Add uint32_t support (#5)

Co-authored-by: David Alonso de la Torre [email protected]

commit sha: 6d071bd8672123501c206383b7cbf217dfe81346

push time in 2 months ago
Activity icon
created branch

davidalo in davidalo/parson create branch add-uint32_t-support

createdAt 2 months ago